Process Steps
Make sure that the system is actually running Apache
$ sudo netstat -pltn | grep 80
tcp 0 0 ::: 80 :::* LISTEN 3432/httpd
$ sudo ss -pltn | grep 80
LISTEN 0 66 :::80 :::* users:((“httpd”,
3432,4), (“httpd”,3435,4), (“httpd”,3436,4), (“httpd”,3437,4), (“httpd”,3438,4))
Know where to look for issues
Common access and error log locations for RHEL/CentOS Cloud Servers
/var/log/apache2/*.log
Common access and error log locations for RHEL/CentOS Cloud Servers
/var/log/httpd/*.log
NOTE : Some customers will place logs in custom locations, if the logs don’t seem to be updating in their stock locations, check the VirtualHost configuration for any other directories and/or log locations they may have specified using the ErrorLog andCustomLog directives
Know how to enable the ‘Debug’logging level
$ sudo grep -i loglevel /etc/{httpd,apache2}/vhost.d/example.site.com
- LogLevel warn
You’ll want to change this to LogLevel Debug on the respective VirtualHost file, and reload Apache. This will generate a tremendous amount of logs, be mindful to switch this back to ‘warn’ after your debugging is complete, or it is possible to fill the VM’s disk with logs.
MaxClients matters alot
The Apache MaxClients directive is set in “MaxClients Reached” /etc/apache2/apache2.conf in Ubuntu/Debian, and /etc/httpd/conf/httpd.conf in RHEL/CentOS. This variable controls the amount of connections that Apache will serve simultaneously. An easy indicator of a high load event, is the access log (s) displaying the warning… At this point it’s best to discuss options with the customer. If this is a normal and expected rise in traffic, proceed with discussing build-out and/or resize options. Or escalate to an L2 or L3 racker for further consultation. You could also analyze the amount of memory that Apache is currently using per child process, and determine (based on the amount of virtual memory available to the VM), raise the limit.
When in doubt, blame
One of the easiest things to get out of the way when troubleshooting an access/site issue, is the .htaccess
file. Customers and developers love to put tons of custom code and snippets in here, that are completely untested and usually just copy and paste from other non-compatible versions of their software, or taken from blogs, etc. Renaming this to .hta or .htaccess.old will quickly take it out of the loop to see if it’s causing the site any harm. Just don’t forget to put it back in and/or address it’s issues afterwards.
Another important topic related to .htaccess is the AllowOverride directive. This is usually found in the Apache VirutalHost configuration file, but duplicate versions can sometimes pop up in the main Apache configuration files as well, so if your changes in the VirutalHost file don’t seem to be making any actual difference, consider grep’ing through the main file(s) as well. If you come across a AllowOverride None declaration in any of the configuration files relative to the domain that your working on, .htaccess will be completely ignored.
Apache Configtest is your friend.. .
In Ubuntu/Debian service apache2 configtest and in RHEL/CentOS /sbin/service httpd configtest
will immediately show any errors or warnings with any configuration changes (Apache main, Apache VirtualHost, etc.) that you may have made, before rendering the site in a down state due to a typo, and put into operation by a restart or reload. Checking your work with configtest is really cheap insurance.
Directory Permissions
The reports of FACLs being cool, are greatly exaggerated. The Apache process(es) are running under needs to have readable permissions to all of the files and directories you plan to be serving content from. You can service this requirement by either group or user, and learning aboutsetgid is a great way to guarntee issues with permissions wont become an issue in the future, due to user error while uploading new content.
SSL Certificates
SSL Certificates are an extremely common request from our users. As of right now, we do not sell SSL certificates. We do provide CSR’s at the users request so that they can take that to an SSL vendor (GoDaddy, Verisign, RapidSSL, etc.) and purchase a certificate. To have an SSL vhost function, you need an SSL certificate, and you need an SSL key that signs it. The way to validate these is with two easy to remember commands:
$ openssl x509 -modulus -in certificate_name.crt -noout Pay close attention to the last 4 to 6 characters in the blurb that follows
$ openssl rsa -modulus -in key_name.key -noout And make sure that they match the last 4 to 6 characters in the blurb that follows
If for whatever reason the moduli do not match, the SSL certificate does not match the SSL key, and Apache will not even restart with these in place. Refer back to the CSR with the customer and find out what got crossed where.
Apache Rewrites
We’re often asked to do simple rewrites, some customers will ask for hundreds to be done. You can safely inform the customer that this is normally developer ended responsibility, for the 4-5 you may come across, there is a cheat-sheet in the wiki: Rewrite Rule Reference
Everything Else… .
If a request sounds odd, refer to the Sphere of Support document. Managed Cloud currently does not service/support Apache Tomact, Solr, suPHP, and/or Passenger beyond Rails installation