LDAP - Installation Process in Linux

LDAP stands for Lightweight Directory Access Protocol. As the name suggests, it is a lightweight client-server protocol for accessing directory services. LDAP runs over TCP/IP or other connection oriented transfer services.

The LDAP server daemon is called Slapd. Slapd supports a variety of different database backends which you can use.They include the primary choice BDB, a high-performance transactional database backend; LDBM, a lightweight DBM based backend.

Pre-Requirements

1) OpenSSL TLS Libraries

2)Database Software-Slapd's primary database backend, BDB, requires Sleepycat Software Berkeley DB

BerkeleyDB install steps:

1. Download BDB from: wget http://download.oracle.com/berkeley-db/db-5.1.25.tar.gz
2. tar xzvf db-5.1.25.tar.gz
3. cd db-5.1.25
4. ./dist/configure --prefix=/custom/BerkeleyDB/5.1.25/
5. make && make install

This install BDB which will be the backend for OpenLDAP.

Now lets install OpenLDAP.

1. Download Openldap at: wget ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.36.tgz
2. cd openldap-2.4.36
3. Set Library paths:

• export CPPFLAGS="-I /custom/BerkeleyDB/5.1.25/include"
• export LD_LIBRARY_PATH="/custom/BerkeleyDB/5.1.25/lib"
• export LDFLAGS="-L/usr/local/lib -L/custom/BerkeleyDB/5.1.25/lib  -R/data/BerkeleyDB/5.1.25/lib"  //note- this step is optional and can be ignored if we are able to run the configure command wthout any error.

   4. ./configure --prefix=/data/openldap/2.4.36
   5. make
   6. make depend
   7. make install

MySQL Install with Binaries

On Unix, to install a compressed tar file binary distribution, unpack it at the installation location you choose (defaultly /usr/local/mysql). 

To install and use a MySQL binary distribution, the basic command sequence looks like this:

shell> groupadd mysql
shell> useradd -r -g mysql mysql
shell> cd /usr/local
shell> tar zxvf /path/to/mysql-VERSION-OS.tar.gz
shell> ln -s full-path-to-mysql-VERSION-OS mysql
shell> cd mysql
shell> chown -R mysql .
shell> chgrp -R mysql .
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data
# Next command is optional
shell> cp support-files/my-medium.cnf /etc/my.cnf
shell> bin/mysqld_safe --user=mysql &
# Next command is optional
shell> cp support-files/mysql.server /etc/init.d/mysql.server

1) Create a mysql user and group

2) Obtain and unpack the mysql tar distribution

3) Run mysql script as mysql user

BIRT Report

BIRT is an Eclipse-based open source reporting system for web applications, especially those based on Java and Java EE. BIRT has two main components: a report designer based on Eclipse, and a runtime component that you can add to your app server. 

With BIRT, you can add a rich variety of reports to your application.

• Lists - The simplest reports are lists of data. As the lists get longer, you can add grouping to organize related data together (orders grouped by customer, products grouped by supplier). If your data is numeric, you can easily add totals, averages and other summaries.


• Charts - Numeric data is much easier to understand when presented as a chart. BIRT provides pie charts, line & bar charts and many more. BIRT charts can be rendered in SVG and support events to allow user interaction.


• Crosstabs - Crosstabs (also called a cross-tabulation or matrix) shows data in two dimensions: sales per quarter or hits per web page.


• Letters & Documents - Notices, form letters, and other textual documents are easy to create with BIRT. Documents can include text, formatting, lists, charts and more.


• Compound Reports - Many reports need to combine the above into a single document. For example, a customer statement may list the information for the customer, provide text about current promotions, and provide side-by-side lists of payments and charges. A financial report may include disclaimers, charts, tables all with extensive formatting that matches corporate color schemes.
  BIRT reports consist of four main parts: data, data transforms, business logic and presentation.
  • Data - Databases, web services, Java objects all can supply data to your BIRT report. BIRT provides JDBC, XML, Web Services, and Flat File support, as well as support for using code to get at other sources of data. BIRT's use of the Open Data Access (ODA) framework allows anyone to build new UI and runtime support for any kind of tabular data. Further, a single report can include data from any number of data sources. BIRT also supplies a feature that allows disparate data sources to be combined using inner and outer joins.
  
  
  • Data Transforms - Reports present data sorted, summarized, filtered and grouped to fit the user's needs. While databases can do some of this work, BIRT must do it for "simple" data sources such as flat files or Java objects. BIRT allows sophisticated operations such as grouping on sums, percentages of overall totals and more.
  
  
  • Business Logic - Real-world data is seldom structured exactly as you'd like for a report. Many reports require business-specific logic to convert raw data into information useful for the user. If the logic is just for the report, you can script it using BIRT's JavaScript support. If your application already contains the logic, you can call into your existing Java code.
  
  
  • Presentation - Once the data is ready, you have a wide range of options for presenting it to the user. Tables, charts, text and more. A single data set can appear in multiple ways, and a single report can present data from multiple data sets.

Python Unit-Testing

The standard workflow of Python Unit-test is:

1. We define your own class derived from unittest.TestCase.
2. Then you fill it with functions that start with ‘test_’.
3. You run the tests by placing unittest.main() in your file, usually at the bottom.

Example 1

import unittestfrom unnecessary_math import multiply
class TestUM(unittest.TestCase):
    def setUp(self):        pass
    def test_numbers_3_4(self):        self.assertEqual( multiply(3,4), 12)
    def test_strings_a_3(self):        self.assertEqual( multiply('a',3), 'aaa')
if __name__ == '__main__':    unittest.main()

In this example,  assertEqual()method is used. 
This method is used to compare the actual and expected values of partcular class execution.

Running Unit Tests
At the bottom of the code we have "if __name__ == '__main__':    unittest.main()" .
This allows us to run all of the test code just by running the file.
Running it with no options is the most terse, and running with a ‘-v’ is more verbose, showing which tests

Output-

> python -m unittest discover simple_example

..

----------------------------------------------------------------------

Ran 2 tests in 0.000s

 

OK

> python -m unittest discover -v simple_example

test_numbers_3_4 (test_um_unittest.TestUM) ... ok

test_strings_a_3 (test_um_unittest.TestUM) ... ok

 

----------------------------------------------------------------------

Ran 2 tests in 0.000s

 

OK

Stored Procedure in MySQL

Stored Procedure Creation

DELIMITER //
 CREATE PROCEDURE <procedure-name>()
   BEGIN
   <query>
   END //
 DELIMITER ;

Syntax :-

The first command is DELIMITER //, which is not related to the stored procedure syntax. 
We use it as we want to pass the  stored procedure to the server as a whole rather than letting mysql tool to interpret each statement at a time.  Following the ENDkeyword, we use the delimiter // to indicate the end of the stored procedure. The last command ( DELIMITER;) changes the delimiter back to the standard one.

We use the CREATE PROCEDURE statement to create a new stored procedure. We specify the name of stored procedure after the CREATE PROCEDURE statement. 

The section between BEGIN and END is called the body of the stored procedure. We put the declarative SQL statements in the body to handle business logic.

Crontab Command in Linux Environment

As i learned the previous week, crontab is useful in setting events that should happen in a periodic basis , automatically if set.
Cron runs in background (Daemon) as default in linux, checks for particular default location "etc/crontab file".

Setting cron has the order as follows :

It has 5 parameters
1)minutes (0-59)
2)hours (0-23)
3)day of month (0-31)
4)month (1-12)
5)day of week (0-7)

Special Characters used :
Asterik(*) – Match all values in the field or any possible value.
Hyphen(-) – To define range.
Slash (/) – 1st field /10 meaning every ten minute or increment of range.
Comma (,) – To separate items.

Commands :
1) Login as root and type" crontab -l" -> This will list the existing root crontab list
2) To view all user entries login as "crontab -u <user> -l"
3) To edit crontab "crontab -e" -> This will deafult edit the currently login user's default crontab ( /tmp/file will be open for editing)
4) To remove currently running / created crontab -> "crontab -r"

Sample setting of a crontab :

• *17 * * 1 date >> ~/testCron.log
  --> This will set cron for every monday(0 index for sunday) 5 p.m. We can pass the value printed to a log file and can trace the execution.
• */10 * * * * date >> ~/testCron.log
  --> This "*/10" part means this will execute for every 10 minutes
  So we can wait for 30 minutes and check it the logfile is update 3 times, for every 10 minutes.

Linux -> Running low on Disk Space -> Mysql Data Binaries

Mysql data binary logs

Mysql binary logs contains records of changesthat affects the database and its  actual data.
It stores delete, update operations, where binary logs allow mysql to recreate events in future.

Since the binary logs contain all data changing events, it is possible to recover changes which occurred after a database dump was created.
 
Issue with these logs

Unfortunately, binary logs do not have a default expiration in mysql.
This means that binary logs are constantly generated, but never removed. This is of concern because the binary logs are not small, and will soon take up a significant amount of space on disk.
The first indication of this is usually " low disk space ".

 Default linux location of these binaries : /usr/local/mysql/data

What i performed to fix the issue is :

• Took dump of mysql database
• Cleared the binaries
• Created the new mysql user, database again and sourced the sql

What actually need to be performed is

 --> Purge old, unneeded binary logs

Steps to perform

1) Type in console : 

 mysql> SHOW SLAVE STATUS \G;

This would show an output like follows :

************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.132.10
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000074
          Read_Master_Log_Pos: 1003370983
                        etc.

 2) If same as here if Slave_IO_state is still in reading state,it is safe to remove any logs created prior to the oldest one being read by replication slaves.

On the Replication master :

mysql> PURGE BINARY LOGS TO 'mysql-bin.000074';

Alternatively, logs prior to a certain date can be removed:

mysql> PURGE BINARY LOGS BEFORE '2011-01-01 00:00:00';

Note
This only takes care of the immediate excess of binary logs.
It does not prevent them from accumulating and resulting in low disk space again. To prevent this, the mysql global variable "expire log days" should be set. This variable determines how many days the binary logs are kept before being automatically purged.
By default it will be set to "0"

3)set expire log days

mysql> SET GLOBAL VARIABLE expire_logs_days=10;

 And also set mysql conf file : /etc/my.cnf
 

[mysqld]  expire_logs_days = 10

Setting this variable will cause MySQL to automatically purge binary logs older than the number of days specified in the variable.


No we can check  the list of binary logs maintained by mysql by running

mysql> SHOW BINARY LOGS;

If the log files are not listed here,then successfully purged.

Reference -->
http://dev.mysql.com/doc/refman/5.0/en/purge-binary-logs.html