RSYSLOG- Server with ADISCON LOGANALYZER

Rsyslog & Adiscon LogAnalyzer using Mysql

Agenda :

Syslog server using the RSYSLOG application (opensource and freeware) and storing the data on a Mysql database and using Adiscon LogAnalyzer to analyze the logs.

Data Flow:-

Pre-requisite:

                               

• RHEL – (RedHat enterprise Linux Server 6.0 +)
• Rsyslog- (Freeware latest version+ mysql modules)
• Adiscon LogAnalyzer- ( Latest version + epel repos)
• Mysql- (default RHEL supported stable release from rhel repo’s) 

Installation :

1.     RHEL & Mysql:

   Install the RHEL 6.0 + version and allocate the 500+ GB disk space to the [/] directory.

1.       Update the OS with latest patches.

a)      Setup YUM update for the Linux.

[root@server ~]# yum install update

b)      Install syslog package, if you do not have it installed.

[root@server ~]# yum install rsyslog

[root@server ~]# yum install rsyslog-mysql*

c)  Install the service httpd & mysql

[root@server ~]#yum install httpd

[root@server ~]#yum install php mysql php-mysql mysql-server rsyslog rsyslog-mysql

 

a)      Download the Adiscon LogAnalyzer and save the same into the Httpd. Main Directory  & apply permissions to the directory.

[root@server ~]# mv loganalyzer  /var/www/html/loganalyzer

[root@server ~]# touch config.php

[root@server ~]# chown apache:apache config.php

[root@server ~]# chmod 777 config.php

b)  Configuring the Mysql for the syslog databse deployment.

# The Database package is already present in the folder>

   /usr/share/doc/rsyslog-mysql-(version)/createDB.sql. Edit the same to change the database name and save a copy for execution into the mysql.

[root@server ~]#mysql -u root -p < /usr/share/doc/rsyslog-mysql-(version)/createDB.sql

Mysql > GRANT ALL PRIVILEGES ON (Databse Name).* TO <Database -User>@localhost IDENTIFIED BY '<User-Password>';

mysql> flush privileges;

mysql> exit.

Test the connectivity to database using the provided username & password

1.     Rsyslog Configuration :

Step 1: Modify the CHKCONFIG for auto start at booting.

[root@server ~]# chkconfig syslog on

Step 2: Modify the configuration of the Rsyslog for enabling the listener & processing the   syslog packets. Details config is also mention below.

[root@server ~]# cd /etc/rsyslog.d/

[root@server ~]# vim Syslog.conf   

Configuration File ( Syslog.conf):

# This configuration has been generated by using the

# rsyslog Configuration Builder which can be found at:

# http://www.rsyslog.com/rsyslog-configuration-builder/

#

# Default Settings

# Load Modules

module(load="imtcp")

module(load="imudp")

module(load="ommysql")

module(load="imuxsock")

#$ActionOmmysqlServerPort 3306

# rsyslog Templates

# rsyslog Input Modules

input(type="imtcp"

         port="514"

         ruleset="FW")

input(type="imudp"

        port="514"

        ruleset="FW")

input(type="imtcp"

         port="516"

         ruleset="SLB")

input(type="imudp"

        port="516"

        ruleset="SLB")

input(type="imtcp"

         port="517"

         ruleset="test")

input(type="imudp"

        port="517"

        ruleset="test")

# rsyslog RuleSets

ruleset(name="FW")

 {

        action(type="ommysql"

                server="localhost"

                serverport="3306"

                db="Syslog"

                uid="<User-name to be entered here>"

                pwd="<Password  to be entered here>")

stop

}

ruleset(name="SLB")

 {

        action(type="ommysql"

                server="localhost"

                serverport="3306"

                db="SyslogSLB"

                uid="<User-name to be entered here>"

                pwd="<Password  to be entered here>")

stop

}

ruleset(name="test")

 {

        action(type="ommysql"

                server="localhost"

                serverport="3306"

                db="Syslogtest"

                uid="<User-name to be entered here>"

                pwd="<Password  to be entered here>")

stop

}

# Default RuleSet 

 

Step 3 : adding of the new device to monitor.

a.       Add the input parameter under < rsyslog Input Modules > on the port the client is transmitting the syslog message along with the protocol.

b.      Add the Ruleset for the processing the input: under<# rsyslog Rulesets>.

Note: This configuration has to be created before the Input and tagged to the input to associate the port to the Ruleset.

c.       Enable the action by forwarding the syslog message under the Ruleset to desired File/database.

Note: we have used the Mysql DB to store the database.

 

1.     Adiscon LogAnalyzer Configuration:

Step 1

If your webserver has write access to the LogAnalyzer folder, you can skip the following step:
Upload the scripts configure.sh and secure.sh from the contrib folder to your webserver, into the same folder where you uploaded the other LogAnalyzer files into. Then set the execution flag to them (chmod +x configure.sh secure.sh).
Now run ./configure.sh, this will create a blank config.php, and will also set write access to everyone to it.
You can of course do this manually if you want.

Step 2

Now open your LogAnalyzer installation in your favourite webbrowser, you will see an error, and you will be pointed to the installation script. The install script will guide you through the LogAnalyzer installation, just follow the instructions.
 

 

Step 3.1

Prerequisites Beginning of installation / welcome site. This is the first page of the installation. It just tells you, that before installing, some file permission have to be checked. Simply click "Next" to start the process.

Step 3.2

Verify the file permissions Here you will see, if the config.php can be written or not. If it cannot be written, you have to repeat the complete Step 2.

 

Step 3.3

Basic Configuration
You can set several basic options here.

·         Number of syslog messages per page = 50 (default)

This is the number of syslog messages displayed on each page. You can increase the value (makes LogAnalyzer slower) or decrease the value (makes it faster).

• Message character limit for the main view = 80 (default)
  Set the number of characters per message which will be shown in the last column of the main view. Full messages can be reviewed by hovering the mouse over it. Many folks prefer to use a setting of "0", which means complete messages will be displayed<
• Show message details popup (default yes) = yes (default). Note that many people find the popups intrusive and prefer to disable them. Use "no" in this case. 

 

 

Step 3.4

Not implemented yet, some real magic may happen here at a later stage ;-) - for now, just skip it.

Step 3.5

Create the first source for syslog messages. This is the most important step. Here, you will configure your first data source, which holds all your syslog data.
Mainly, you have to choose a "Name of the Source" and a "Source Type". The name will be displayed later in a drop-down menu with which you choose your active syslog source. The "Source Type" can be a file, a MySQL database or the PHP PDO which supports different database types like mssql, PostgreSQL, odbc, oracle or even ibm db2.

If you choose the diskfile, you have to provide the following information:

·         Logline Type = Syslog / Rsyslog (default) or Adiscon WinSyslog

This tells LogAnalyzer, how the lines look like. This is necessary for show the log messages properly.

·         Syslog File = /var/log/syslog (default)

This is the position of the logfile in your file system.

If you choose MySQL native as data source, following information is needed:

• Table Type = monitorware (default)

This is the table layout. Currently, you can use "monitorware" or "syslogng". For more details see "Note on MySQL Databases" below.

·         Database Host = localhost (default)

This is the host, where the database is located. By default this is localhost. You can specify any other host if necessary.

·         Database Name = loganalyzer (default)

The name of the database you want to use.

·         Database Tablename = systemevents (default)

This is the name of the table in which the data is stored. The default tablename corresponds to the tables created with the MonitorWare Line of products.

·         Database User = user (default)

The username for the database.

·         Database Password = not set by default

The password for the username.

·         Enable Row Counting = No (default)

If configured to "Yes", the amount of rows in the table will be counted with every query, giving you the total records for your search, though having a lot of impact on your system when using a very large database. If configured to "No", the rows will not be counted, providing you a lot more performance.

If you choose Database (PDO), the following has to be defined:

·         Database Storage Engine = MySQL Server (default)

Choose the engine of the database you are using. These databases are supported: MySQL Server, Microsoft SQL Server, ODBC Database Connection, PostgreSQL, Oracle Call Interface, IBM DB2, Firebird/Interbase 6, IBM Informix Dynamic Server, SQLite 2.

·         Table Type = monitorware (default)

This is the table layout. Currently, you can use "monitorware" or "syslogng". For more details see "Note on MySQL Databases" below.

·         Database Host = localhost (default)

This is the host, where the database is located. By default this is localhost. You can specify any other host if necessary.

·         Database Name = loganalyzer (default)

The name of the database you want to use.

·         Database Tablename = systemevents (default)

This is the name of the table in which the data is stored. The default tablename corresponds to the tables created with the MonitorWare Line of products.

·         Database User = user (default)

The username for the database.

·         Database Password = not set by default

The password for the username.

·         Enable Row Counting = No (default)

If configured to "Yes", the amount of rows in the table will be counted with every query, giving you the total records for your search, though having a lot of impact on your system when using a very large database. If configured to "No", the rows will not be counted, providing you a lot more performance. 

Step 4: Finish.

      

Now you will be automaticall redirected to the Adiscon LogAnalyzer if not open the browser and type the URL manually.

Http:// 192.168.0.1/<Loganalyzer dir name>/src

      

THE END.