The number of servers in our farm is continuing to grow. It’s becoming more and more difficult to monitor them all as closely as we would like. We decided that it’s time to set up as centralized location for log files to keep a closer eye on everything and allow us to easily develop our own reports and triggers against the logs. For this, we will be using rsyslog with a 3rd party program, LogAnalyzer. For the purposes of this document, we will assume that you already have a MySQL database configured running on a separate server.
Configure the Server
1) To begin, make sure your system is up to date so there are no package discrepancies later.
2) Your system should have rsyslog installed and running already, but we will need a few additional components along with the mysql client.
yum install syslog-* mysql
3) Make sure the service is started and enabled on boot
service rsyslog start chkconfig rsyslog on
4) Next, we will load the new tables to the database. Part of the installs from step 2 provided the table structures for the new database. By default, the database name is “syslog”. If you would like to change this, please open the file in a text editor first. The database name is defined on the first two lines.
cd /usr/share/doc/rsyslog-mysql-5.8.10 vim createDB.sql mysql -h 192.168.0.23 -u syslog -p syslog < createDB.sql
5) To configure rsyslog to use the new MySQL database, we will need to make a few edits to /etc/rsyslog.conf.
Uncomment the following module to allow the server to receive syslog messages
# Provides TCP syslog reception $ModLoad imtcp $InputTCPServerRun 5822
At the very bottom, add the MySQL module with your connection details, then save/exit.
$ModLoad ommysql *.* :ommysql:syslogdbHost,syslogdb,sysloguser,YourPassword
6) Restart the syslog service. You should see events start appearing in the database now
service rsyslog restart
7) Next, we need to install and start Apache and PHP for the LogAnalyzer software
yum install httpd php php-mysql php-gd service httpd start chkconfig httpd on
8) Download the latest LogAnalyzer. At the time of writing, the latest version was 4.1.2
cd ~ wget http://download.adiscon.com/loganalyzer/loganalyzer-4.1.2.tar.gz tar xf loganalyzer-4.1.2.tar.gz cd loganalyzer-4.1.2 cp -r src/* /var/www/html/ cp -r contrib/* /var/www/html/
9) Configure and set some permissions
cd /var/www/html/ chmod +x configure.sh secure.sh ./configure.sh
10) To complete the installation, open your web browser and go to your server’s address. For me, it’s http://192.168.0.29/. During the installation, make sure to check “yes” to “enable user database” and enter your database credentials. Remember also that schema and table names are case-sensitive!
Configure the Clients
1) Open /etc/rsyslog.conf and add make the following two changes
We prefer to keep the fully qualified domain name. This must be the first loaded module, it’s a little picky… Add it right under the “MODULES” comment.
#### MODULES #### $PreserveFQDN on
Next, tell it to send all logs to our logging server. You’ll notice the two @@ symbols. This will force it to use TCP. if you prefer to use UDP, you can replace them with a single @.
2) Restart rsyslog and you should be good to go!
service rsyslog restart
That’s it, you’re all set! Happy monitoring.