I’ve run into an issue a couple of times where some web applications on my server have broken following a restart of Apache when the application in question calls external programs..
What seems to happen is that when an administrator restarts Apache during general maintenance of that server, Apache picks up some of the unwanted environmental settings from the root user account, in particular the variable HOME ends up getting set to the home directory of the root user account (/root).
Generally it won’t be an issue for web applications, but if they call an external application (in my case, Git), that external application may use the HOME environment to try and read or write configuration files.
# tail -n1 error.log fatal: unable to access '/root/.config/git/config': Permission denied
In my case, Git kept dying with a fatal error, which lead to a very confused sysadmin wondering why a process running as Apache is trying to read from the root user’s account…
By looking at the environmental settings for the Apache worker processes, we can see what’s happening. After a normal boot, the environmental variables look something like the below:
# ps aux | grep httpd root 10173 0.0 1.6 27532 8496 ? Ss 22:42 0:00 /usr/sbin/httpd apache 10175 0.1 2.8 37560 14692 ? S 22:42 0:01 /usr/sbin/httpd apache 10176 0.1 2.8 37836 14952 ? S 22:42 0:01 /usr/sbin/httpd apache 10177 0.1 2.8 37332 14876 ? S 22:42 0:01 /usr/sbin/httpd apache 10178 0.1 2.8 37560 14692 ? S 22:42 0:01 /usr/sbin/httpd # cat /proc/10175/environ TERM=dumbPATH=/sbin:/usr/sbin:/bin:/usr/binPWD=/LANG=CSHLVL=2_=/usr/sbin/httpd
Because Apache has been started by init, it has a nice clean environment. But after a restart by the root user, it’s clear that some cruft from the root user account has been pulled into the application environment variables:
# cat /proc/10175/environ HOSTNAME=localhostSHELL=/bin/bashTERM=xtermHISTSIZE=1000USER=root: MAIL=/var/spool/mail/rootPATH=/sbin:/usr/sbin:/bin:/usr/bin INPUTRC=/etc/inputrcPWD=/rootLANG=CSHLVL=3HOME=/rootLOGNAME=root LESSOPEN=|/usr/bin/lesspipe.sh %sG_BROKEN_FILENAMES=1_=/usr/sbin/httpd
Because of these settings, external programs relying on the value of HOME will try to read/write to a directory that they aren’t permitted to use.
Debian-based systems fix this issue by unsetting certain environmentals (including HOME) in the bootscript for Apache, based on the rules in /etc/apache2/envvars.
To fix the issue on a RHEL/CentOS host, you can instead just append a replacement HOME setting into /etc/sysconfig/httpd. This particular configuration file is read at server startup and isn’t overwritten when Apache gets upgraded.
cat >> /etc/sysconfig/httpd << "EOF" # Correct Apache's home directory HOME=/var/www EOF
Following a restart, Apache should now show the correct HOME environmental variable and your application should function as expected.