[clamav-users] possible rar issues when files have special characters

iulian stan iulian at sphere.ro
Mon Oct 5 11:33:17 UTC 2020 

Again me, 

Ups, now i found this. 

Micah, can you confirm it's the same issue that you fixed in 0.103 ? 

"

The UnRAR library requires the character-classification locale to be set
to the empty string "" so it will be set according to the environment
variables, as seeen in the rar.cpp example application `main()`.

Without this, extracting RAR archives containing unicode filenames on
non-Windows, non-macOS operating systems may fail
"

---

Best regards,
Iulian 

On 2020-10-05 14:13, iulian stan via clamav-users wrote:

> Hi all, 
> 
> I've filled a bug: https://bugzilla.clamav.net/show_bug.cgi?id=12621 
> 
> The problem is present on Gentoo(installed via emerge) and Ubuntu 18.04.1 LTS(installed via apt-get). If others can test on other platforms will be great. 
> 
> Funny thing: on Windows ( clamav-0.102.4-win-x64-portable.zip) it is working ok.
> 
> ---
> Best regards,
> Iulian 
> 
> On 2020-10-04 16:40, Iulian Stan wrote:
> 
>> Hello,  
>> 
>> I didn't had time to investigate too much since is weekend and family will be really unhappy:)) 
>> Since the whole investigation was made on the phone i will be brief. 
>> 
>> --leave-temps doesn't provide any clue but debug clarifies the problem. 
>> 
>> Unfortunately we face a bug(i will also look tomorrow for what is reported already). 
>> Simple put when special characters are set the name of the file(including file extension) is truncated. 
>> 
>> With special caracter: 
>> 
>> LibClamAV debug: Checking realpath of just.rar
>> LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)
>> LibClamAV debug: Recognized RAR file
>> LibClamAV debug: cache_check: 2c04496b1308e6349e3726f91e156235 is negative
>> LibClamAV debug: in scanrar()
>> unrar_open: Comments are not present in this archive.
>> unrar_open: Volume attribute (archive volume):              no
>> unrar_open: Archive comment present:                        no
>> unrar_open: Archive lock attribute:                         no
>> unrar_open: Solid attribute (solid archive):                no
>> unrar_open: New volume naming scheme ('volname.partN.rar'): yes
>> unrar_open: Authenticity information present (obsolete):    no
>> unrar_open: Recovery record present:                        no
>> unrar_open: Block headers are encrypted:                    no
>> unrar_open: First volume (set only by RAR 3.0 and later):   no
>> unrar_open: Opened archive: /home/iulian/viruses/1/just.rar
>> unrar_peek_file_header:   Name:          CONSILIERE PLAT
>> unrar_peek_file_header:   Directory?:    0
>> unrar_peek_file_header:   Target Dir:    0
>> unrar_peek_file_header:   RAR Version:   50
>> unrar_peek_file_header:   Packed Size:   5
>> unrar_peek_file_header:   Unpacked Size: 5
>> LibClamAV debug: RAR: CONSILIERE PLAT, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1
>> LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLAT:5:5:0:1:1001993670:(nil)
>> LibClamAV debug: RAR: Extracting file: CONSILIERE PLAT to /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmp
>> unrar_extract_file: Extracted file to: /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmp
>> LibClamAV debug: RAR: Extraction complete.  Scanning now...
>> LibClamAV debug: in cli_magic_scandesc (reclevel: 1/16)
>> LibClamAV debug: Small data (5 bytes)
>> LibClamAV debug: cli_magic_scandesc: returning 0  at line 4057 (no post, no cache)
>> unrar_retcode: No more files in archive.
>> LibClamAV debug: RAR: No more files in archive.
>> LibClamAV debug: RAR: Exit code: 0
>> LibClamAV debug: hashtab: Freeing hashset, elements: 0, capacity: 0
>> LibClamAV debug: cli_magic_scandesc: returning 0  at line 3202
>> LibClamAV debug: cache_add: 2c04496b1308e6349e3726f91e156235 (level 0)
>> /home/iulian/viruses/1/just.rar: OK
>> LibClamAV debug: Cleaning up phishcheck
>> LibClamAV debug: Freeing phishcheck struct
>> LibClamAV debug: Phishcheck cleaned up 
>> 
>> Without special characters: 
>> 
>> LibClamAV debug: Checking realpath of anothertest.rar
>> LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)
>> LibClamAV debug: Recognized RAR file
>> LibClamAV debug: cache_check: bbe25db3191912601ee2b12860c99627 is negative
>> LibClamAV debug: in scanrar()
>> unrar_open: Comments are not present in this archive.
>> unrar_open: Volume attribute (archive volume):              no
>> unrar_open: Archive comment present:                        no
>> unrar_open: Archive lock attribute:                         no
>> unrar_open: Solid attribute (solid archive):                no
>> unrar_open: New volume naming scheme ('volname.partN.rar'): yes
>> unrar_open: Authenticity information present (obsolete):    no
>> unrar_open: Recovery record present:                        no
>> unrar_open: Block headers are encrypted:                    no
>> unrar_open: First volume (set only by RAR 3.0 and later):   no
>> unrar_open: Opened archive: /home/iulian/viruses/1/anothertest.rar
>> unrar_peek_file_header:   Name:          CONSILIERE PLATA_Pdf.exe
>> unrar_peek_file_header:   Directory?:    0
>> unrar_peek_file_header:   Target Dir:    0
>> unrar_peek_file_header:   RAR Version:   50
>> unrar_peek_file_header:   Packed Size:   5
>> unrar_peek_file_header:   Unpacked Size: 5
>> LibClamAV debug: RAR: CONSILIERE PLATA_Pdf.exe, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1
>> LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLATA_Pdf.exe:5:5:0:1:1001993670:(nil)
>> LibClamAV debug: FP SIGNATURE: bbe25db3191912601ee2b12860c99627:95:Archived_EXE.UNOFFICIAL/home/iulian/viruses/1/anothertest.rar: Archived_EXE.UNOFFICIAL FOUND
>> LibClamAV debug: RAR: Exit code: 1
>> LibClamAV debug: cli_magic_scandesc: returning 1  at line 3202
>> LibClamAV debug: Cleaning up phishcheck
>> LibClamAV debug: Freeing phishcheck struct
>> LibClamAV debug: Phishcheck cleaned up 
>> 
>> Best regads, 
>> Iulian 
>> 
>> Sent from my Samsung Galaxy smartphone. 
>> 
>> Hello, I didn't had time to investigate too much since is weekend and family will be really unhappy:))Since the whole investigation was made on the phone i will be brief.--leave-temps doesn't provide any clue but debug clarifies the problem.Unfortunately we face a bug(i will also look tomorrow for what is reported already).Simple put when special characters are set the name of the file(including file extension) is truncated.With special caracter:LibClamAV debug: Checking realpath of just.rarLibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)LibClamAV debug: Recognized RAR fileLibClamAV debug: cache_check: 2c04496b1308e6349e3726f91e156235 is negativeLibClamAV debug: in scanrar()unrar_open: Comments are not present in this archive.unrar_open: Volume attribute (archive volume):              nounrar_open: Archive comment present:                        nounrar_open: Archive lock attribute:                         nounrar_open: Solid attribute (solid archive):               
nounrar_open: New volume naming scheme ('volname.partN.rar'): yesunrar_open: Authenticity information present (obsolete):    nounrar_open: Recovery record present:                        nounrar_open: Block headers are encrypted:                    nounrar_open: First volume (set only by RAR 3.0 and later):   nounrar_open: Opened archive: /home/iulian/viruses/1/just.rarunrar_peek_file_header:   Name:          CONSILIERE PLATunrar_peek_file_header:   Directory?:    0unrar_peek_file_header:   Target Dir:    0unrar_peek_file_header:   RAR Version:   50unrar_peek_file_header:   Packed Size:   5unrar_peek_file_header:   Unpacked Size: 5LibClamAV debug: RAR: CONSILIERE PLAT, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLAT:5:5:0:1:1001993670:(nil)LibClamAV debug: RAR: Extracting file: CONSILIERE PLAT to /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmpunrar_extract_file: Extracted file to:
/tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmpLibClamAV debug: RAR: Extraction complete.  Scanning now...LibClamAV debug: in cli_magic_scandesc (reclevel: 1/16)LibClamAV debug: Small data (5 bytes)LibClamAV debug: cli_magic_scandesc: returning 0  at line 4057 (no post, no cache)unrar_retcode: No more files in archive.LibClamAV debug: RAR: No more files in archive.LibClamAV debug: RAR: Exit code: 0LibClamAV debug: hashtab: Freeing hashset, elements: 0, capacity: 0LibClamAV debug: cli_magic_scandesc: returning 0  at line 3202LibClamAV debug: cache_add: 2c04496b1308e6349e3726f91e156235 (level 0)/home/iulian/viruses/1/just.rar: OKLibClamAV debug: Cleaning up phishcheckLibClamAV debug: Freeing phishcheck structLibClamAV debug: Phishcheck cleaned upWithout special characters:LibClamAV debug: Checking realpath of anothertest.rarLibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)LibClamAV debug: Recognized RAR fileLibClamAV debug: cache_check:
bbe25db3191912601ee2b12860c99627 is negativeLibClamAV debug: in scanrar()unrar_open: Comments are not present in this archive.unrar_open: Volume attribute (archive volume):              nounrar_open: Archive comment present:                        nounrar_open: Archive lock attribute:                         nounrar_open: Solid attribute (solid archive):                nounrar_open: New volume naming scheme ('volname.partN.rar'): yesunrar_open: Authenticity information present (obsolete):    nounrar_open: Recovery record present:                        nounrar_open: Block headers are encrypted:                    nounrar_open: First volume (set only by RAR 3.0 and later):   nounrar_open: Opened archive: /home/iulian/viruses/1/anothertest.rarunrar_peek_file_header:   Name:          CONSILIERE PLATA_Pdf.exeunrar_peek_file_header:   Directory?:    0unrar_peek_file_header:   Target Dir:    0unrar_peek_file_header:   RAR Version:   50unrar_peek_file_header:   Packed Size:  
5unrar_peek_file_header:   Unpacked Size: 5LibClamAV debug: RAR: CONSILIERE PLATA_Pdf.exe, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLATA_Pdf.exe:5:5:0:1:1001993670:(nil)LibClamAV debug: FP SIGNATURE: bbe25db3191912601ee2b12860c99627:95:Archived_EXE.UNOFFICIAL/home/iulian/viruses/1/anothertest.rar: Archived_EXE.UNOFFICIAL FOUNDLibClamAV debug: RAR: Exit code: 1LibClamAV debug: cli_magic_scandesc: returning 1  at line 3202LibClamAV debug: Cleaning up phishcheckLibClamAV debug: Freeing phishcheck structLibClamAV debug: Phishcheck cleaned upBest regads,IulianSent from my Samsung Galaxy smartphone. 
>> 
>> -------- Original message -------- 
>> From: "G.W. Haywood via clamav-users" <clamav-users at lists.clamav.net> 
>> Date: 10/4/20 12:27 (GMT+02:00) 
>> To: iulian stan via clamav-users <clamav-users at lists.clamav.net> 
>> Cc: "G.W. Haywood" <clamav at jubileegroup.co.uk> 
>> Subject: Re: [clamav-users] possible rar issues when files have special   characters 
>> Hi there,
>> 
>> On Sun, 4 Oct 2020, iulian stan via clamav-users wrote:
>> 
>>> I know that relying on the file extension is not perfect but i will
>>> say it is covering most of the threats.
>> 
>> Understood, a pragmatic approach.
>> 
>>> Anyhow my raised question was about: Why .exe is not detected when
>>> the file inside archive has a special character?  This problem is
>>> manifesting only with RAR.  For files which don't have special
>>> character RAR is behaving as expected.
>> 
>> Good question.  Perhaps if you use the --leave-temps option and
>> inspect the temporary files left after scanning it might shed some
>> light on the issue.  Have you checked the ClamAV Bugzilla issues to
>> see if there's anything similar mentioned?
>> 
>> Does the same thing also happen if you use clamdscan instead?
>> 
>> Can you simply block all .rar files?  I do that for mail, but I don't
>> generally scan filesystems at all.
>> 
>> -- 
>> 
>> 73,
>> Ged.
>> 
>> _______________________________________________
>> 
>> clamav-users mailing list
>> clamav-users at lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>> 
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>> 
>> http://www.clamav.net/contact.html#ml 
>> -------- Original message --------From: "G.W. Haywood via clamav-users" <clamav-users at lists.clamav.net> Date: 10/4/20  12:27  (GMT+02:00) To: iulian stan via clamav-users <clamav-users at lists.clamav.net> Cc: "G.W. Haywood" <clamav at jubileegroup.co.uk> Subject: Re: [clamav-users] possible rar issues when files have special
>> characters Hi there,On Sun, 4 Oct 2020, iulian stan via clamav-users wrote:> I know that relying on the file extension is not perfect but i will> say it is covering most of the threats.Understood, a pragmatic approach.> Anyhow my raised question was about: Why .exe is not detected when> the file inside archive has a special character?  This problem is> manifesting only with RAR.  For files which don't have special> character RAR is behaving as expected.Good question.  Perhaps if you use the --leave-temps option andinspect the temporary files left after scanning it might shed somelight on the issue.  Have you checked the ClamAV Bugzilla issues tosee if there's anything similar mentioned?Does the same thing also happen if you use clamdscan instead?Can you simply block all .rar files?  I do that for mail, but I don'tgenerally scan filesystems at all.-- 73,Ged._______________________________________________clamav-users mailing
listclamav-users at lists.clamav.nethttps://lists.clamav.net/mailman/listinfo/clamav-usersHelp us build a comprehensive ClamAV guide:https://github.com/vrtadmin/clamav-faqhttp://www.clamav.net/contact.html#ml
> 
> _______________________________________________
> 
> clamav-users mailing list
> clamav-users at lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
> 
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20201005/20dffb1d/attachment.htm>

More information about the clamav-users mailing list