> here are the logs from 10:01 AM Jul 13:
> Jul 13 10:01:02 storm freshclam[3930506]: Database test passed.
> Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version: 26230, sigs: 3995778, f-level: 63, builder: raynman)
> Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version: 26230, sigs: 3995778, f-level: 63, builder: raynman)
> ...
> ps -auwx|grep freshclam
> clamav      3818  0.0  0.0  28952 12864 ?        Ss   12:00   0:00 /usr/bin/freshclam -d --foreground=true

The logs contain a lot of duplicated lines.  Maybe you have both a
line like

StandardOutput=syslog

in your freshclam.service and *also* a line like

LogSyslog yes

in your freshclam.conf (or whatever passes for freshclam.conf in these
screwy RedHat systems).  Well, you want one or the other but not both.
I'd suggest commenting out the "LogSyslog yes" line and restarting the
freshclam daemon.

I didn't have both set to yes but now I commented out LogSyslog and set freshclam to log to its own log:
grep -v ^\# /etc/freshclam.conf | grep  .
DatabaseDirectory /var/lib/clamav
UpdateLogFile /var/log/freshclam.log
DatabaseOwner clamav
DatabaseMirror database.clamav.net
ConnectTimeout 60
ReceiveTimeout 60
SafeBrowsing no

Side note I did have af few of these enabled but I believe the unofficial sigs program has them.
#DatabaseCustomURL http://www.securiteinfo.com/get/signatures/xxx/securiteinfo.hdb
#DatabaseCustomURL http://www.securiteinfo.com/get/signatures/xxx/securiteinfo.ign2
 
Looks like it was either the update or the reboot.  An easy way to
find out would be to just reboot.

Perhaps next week as we have summer session and some researchers logged in

>> Assuming that we can believe the timestamps, then any problems that
>> arose from ownership by the clamupdate user/group had already happened
>> at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
>> which caused them.
>>
>> Is this the first time that clamav-unofficial-sigs.sh ran?
>
> No it's been running all the time.

I think we're confusing each other.  The clamav-unofficial-sigs.sh
script doesn't run like a daemon runs.  The script is started by
something like a cron entry; it updates the configured databases if
needed, then stops.  The unofficial update script only updates (or
should only update) the third-party signature database files, that is
everything except 'main', 'daily' and 'bytecode'.  I meant was it at
12:14 that the clamav-unofficial-sigs.sh script ran?  Presumably it's
logging its activities somewhere, do you have that log?  The log
location will be set in the configuration for the unofficial script.

Uploaded at https://storm.cis.fordham.edu/~rkudyba/clamav-unofficial-sigs.log
 
  Something changed the
permissions on the directory /var/lib/clamav/, freshclam can no longer
create the temporary directory it needs for the update.  I guess the
unofficial update script disagrees with freshclam about the user and
group for the database directory.  I believe freshclam is running as
user clamupdate, group clamupdate (983:979) but you have the ownership
on the directory set to user clamav, group clamav (985:981). 

Right which is why I also tried added user clamav to the clamupdate group.
 
Note
that the only things which need to be able to write to the directory
are freshclam and the unofficial update script.  These two must be
configured to use the same user:group (or at least so that they can
both write into the directory).  Find out where these things are
configured and set them the same. 

grep -i owner /etc/freshclam.conf (comments removed)
DatabaseOwner clamav

grep -i user /etc/clamav-unofficial-sigs/user.conf
user_configuration_complete="yes"
clam_user="clamav"

grep -i user /etc/clamav-unofficial-sigs/user.conf
user_configuration_complete="yes"
clam_user="clamav"
 
If RedHat updates are setting the
configuration of the freshclam daemon to use clamupdate:clamupdate it
would explain why this happens after an update. 

AFAIK, you have to manually run something like rpmconf -a to do force a new config file.
 
In that case it may
be better to settle on clamupdate:clamupdate for both freshclam and
the unofficial script. 

I'm starting to believe the same. This is how it's set on another server I oversee and no issues.
 
It's also possible that you have something in
the startup which is setting the directory user:group too, we'll take
a look at that later if need be.

I believe it's this file:
cat /usr/lib/systemd/system/clamav-freshclam.service
[Unit]
Description=ClamAV virus database updater
Documentation=man:freshclam(1) man:freshclam.conf(5) https://www.clamav.net/documents
# If user wants it run from cron, don't start the daemon.
ConditionPathExists=!/etc/cron.d/clamav-update
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/freshclam -d --foreground=true

[Install]
WantedBy=multi-user.target
 
After all this you may have to manually set the user/group IDs of
/var/lib/clamav/ and the files in there (you can do that with the
'chown' and 'chgrp' commands) and you'll need to restart the freshclam
daemon (again).

If/when this happens again I may just do that. 

Finally when the dust has settled and you're happy that it isn't going
to break again you can stop the cron job logging the permissions.  But
leave it running until you're sure. 

noted.