Sunday, September 21, 2014

LDAP - Installation Process in Linux

LDAP stands for Lightweight Directory Access Protocol. As the name suggests, it is a lightweight client-server protocol for accessing directory services. LDAP runs over TCP/IP or other connection oriented transfer services.


The LDAP server daemon is called Slapd. Slapd supports a variety of different database backends which you can use.They include the primary choice BDB, a high-performance transactional database backend; LDBM, a lightweight DBM based backend.

Pre-Requirements
1) OpenSSL TLS Libraries
2)Database Software-Slapd's primary database backend, BDB, requires Sleepycat Software Berkeley DB

BerkeleyDB install steps:

  1. Download BDB from: wget http://download.oracle.com/berkeley-db/db-5.1.25.tar.gz
  2. tar xzvf db-5.1.25.tar.gz
  3. cd db-5.1.25
  4. ./dist/configure --prefix=/custom/BerkeleyDB/5.1.25/
  5. make && make install
This install BDB which will be the backend for OpenLDAP.

Now lets install OpenLDAP.

  1. Download Openldap at: wget ftp://ftp.openldap.org/pub/OpenLDAP/openldap-release/openldap-2.4.36.tgz
  2. cd openldap-2.4.36
  3. Set Library paths:

  • export CPPFLAGS="-I /custom/BerkeleyDB/5.1.25/include"
  • export LD_LIBRARY_PATH="/custom/BerkeleyDB/5.1.25/lib"
  • export LDFLAGS="-L/usr/local/lib -L/custom/BerkeleyDB/5.1.25/lib  -R/data/BerkeleyDB/5.1.25/lib"  //note- this step is optional and can be ignored if we are able to run the configure command wthout any error.
   4. ./configure --prefix=/data/openldap/2.4.36
   5. make
   6. make depend
   7. make install



Monday, September 8, 2014

MySQL Install with Binaries

On Unix, to install a compressed tar file binary distribution, unpack it at the installation location you choose (defaultly /usr/local/mysql). 


To install and use a MySQL binary distribution, the basic command sequence looks like this:
shell> groupadd mysql
shell> useradd -r -g mysql mysql
shell> cd /usr/local
shell> tar zxvf /path/to/mysql-VERSION-OS.tar.gz
shell> ln -s full-path-to-mysql-VERSION-OS mysql
shell> cd mysql
shell> chown -R mysql .
shell> chgrp -R mysql .
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data
# Next command is optional
shell> cp support-files/my-medium.cnf /etc/my.cnf
shell> bin/mysqld_safe --user=mysql &
# Next command is optional
shell> cp support-files/mysql.server /etc/init.d/mysql.server

1) Create a mysql user and group
2) Obtain and unpack the mysql tar distribution
3) Run mysql script as mysql user

BIRT Report

BIRT is an Eclipse-based open source reporting system for web applications, especially those based on Java and Java EE. BIRT has two main components: a report designer based on Eclipse, and a runtime component that you can add to your app server. 


With BIRT, you can add a rich variety of reports to your application.
  • Lists - The simplest reports are lists of data. As the lists get longer, you can add grouping to organize related data together (orders grouped by customer, products grouped by supplier). If your data is numeric, you can easily add totals, averages and other summaries.

  • Charts - Numeric data is much easier to understand when presented as a chart. BIRT provides pie charts, line & bar charts and many more. BIRT charts can be rendered in SVG and support events to allow user interaction.

  • Crosstabs - Crosstabs (also called a cross-tabulation or matrix) shows data in two dimensions: sales per quarter or hits per web page.

  • Letters & Documents - Notices, form letters, and other textual documents are easy to create with BIRT. Documents can include text, formatting, lists, charts and more.

  • Compound Reports - Many reports need to combine the above into a single document. For example, a customer statement may list the information for the customer, provide text about current promotions, and provide side-by-side lists of payments and charges. A financial report may include disclaimers, charts, tables all with extensive formatting that matches corporate color schemes.
    BIRT reports consist of four main parts: data, data transforms, business logic and presentation.
    • Data - Databases, web services, Java objects all can supply data to your BIRT report. BIRT provides JDBC, XML, Web Services, and Flat File support, as well as support for using code to get at other sources of data. BIRT's use of the Open Data Access (ODA) framework allows anyone to build new UI and runtime support for any kind of tabular data. Further, a single report can include data from any number of data sources. BIRT also supplies a feature that allows disparate data sources to be combined using inner and outer joins.

    • Data Transforms - Reports present data sorted, summarized, filtered and grouped to fit the user's needs. While databases can do some of this work, BIRT must do it for "simple" data sources such as flat files or Java objects. BIRT allows sophisticated operations such as grouping on sums, percentages of overall totals and more.

    • Business Logic - Real-world data is seldom structured exactly as you'd like for a report. Many reports require business-specific logic to convert raw data into information useful for the user. If the logic is just for the report, you can script it using BIRT's JavaScript support. If your application already contains the logic, you can call into your existing Java code.

    • Presentation - Once the data is ready, you have a wide range of options for presenting it to the user. Tables, charts, text and more. A single data set can appear in multiple ways, and a single report can present data from multiple data sets.

Sunday, September 7, 2014

Python Unit-Testing

The standard workflow of Python Unit-test is:

1. We define your own class derived from unittest.TestCase.
2. Then you fill it with functions that start with ‘test_’.
3. You run the tests by placing unittest.main() in your file, usually at the bottom.


Example 1
import unittestfrom unnecessary_math import multiply
class TestUM(unittest.TestCase):
    def setUp(self):        pass
    def test_numbers_3_4(self):        self.assertEqual( multiply(3,4), 12)
    def test_strings_a_3(self):        self.assertEqual( multiply('a',3), 'aaa')
if __name__ == '__main__':    unittest.main()
In this example,  assertEqual()method is used
This method is used to compare the actual and expected values of partcular class execution.

Running Unit Tests
At the bottom of the code we have "if __name__ == '__main__':    unittest.main()" .
This allows us to run all of the test code just by running the file.
Running it with no options is the most terse, and running with a ‘-v’ is more verbose, showing which tests

Output-
> python -m unittest discover simple_example
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s
 
OK
> python -m unittest discover -v simple_example
test_numbers_3_4 (test_um_unittest.TestUM) ... ok
test_strings_a_3 (test_um_unittest.TestUM) ... ok
 
----------------------------------------------------------------------
Ran 2 tests in 0.000s
 
OK

Saturday, September 6, 2014

Stored Procedure in MySQL

Stored Procedure Creation

DELIMITER //
 CREATE PROCEDURE <procedure-name>()
   BEGIN
   <query>
   END //
 DELIMITER ;

Syntax :-

The first command is DELIMITER //, which is not related to the stored procedure syntax. 
We use it as we want to pass the  stored procedure to the server as a whole rather than letting mysql tool to interpret each statement at a time.  Following the ENDkeyword, we use the delimiter // to indicate the end of the stored procedure. The last command ( DELIMITER;) changes the delimiter back to the standard one.

We use the CREATE PROCEDURE statement to create a new stored procedure. We specify the name of stored procedure after the CREATE PROCEDURE statement. 


The section between BEGIN and END is called the body of the stored procedure. We put the declarative SQL statements in the body to handle business logic.

Monday, August 4, 2014

Crontab Command in Linux Environment

As i learned the previous week, crontab is useful in setting events that should happen in a periodic basis , automatically if set.
Cron runs in background (Daemon) as default in linux, checks for particular default location "etc/crontab file".

Setting cron has the order as follows :

It has 5 parameters
1)minutes (0-59)
2)hours (0-23)
3)day of month (0-31)
4)month (1-12)
5)day of week (0-7)

Special Characters used :
Asterik(*) – Match all values in the field or any possible value.
Hyphen(-) – To define range.
Slash (/) – 1st field /10 meaning every ten minute or increment of range.
Comma (,) – To separate items.

Commands :
1) Login as root and type" crontab -l" -> This will list the existing root crontab list
2) To view all user entries login as "crontab -u <user> -l"
3) To edit crontab "crontab -e" -> This will deafult edit the currently login user's default crontab ( /tmp/file will be open for editing)
4) To remove currently running / created crontab -> "crontab -r"

Sample setting of a crontab :
  • *17 * * 1 date >> ~/testCron.log
    --> This will set cron for every monday(0 index for sunday) 5 p.m. We can pass the value printed to a log file and can trace the execution.
  • */10 * * * * date >> ~/testCron.log
    --> This "*/10" part means this will execute for every 10 minutes
    So we can wait for 30 minutes and check it the logfile is update 3 times, for every 10 minutes.


 
 
 
 
 
 

Tuesday, July 1, 2014

Linux -> Running low on Disk Space -> Mysql Data Binaries

Mysql data binary logs

Mysql binary logs contains records of changesthat affects the database and its  actual data.
It stores delete, update operations, where binary logs allow mysql to recreate events in future.

Since the binary logs contain all data changing events, it is possible to recover changes which occurred after a database dump was created.
 
Issue with these logs

Unfortunately, binary logs do not have a default expiration in mysql.
This means that binary logs are constantly generated, but never removed. This is of concern because the binary logs are not small, and will soon take up a significant amount of space on disk.
The first indication of this is usually " low disk space ".

 Default linux location of these binaries : /usr/local/mysql/data








What i performed to fix the issue is :
  • Took dump of mysql database
  • Cleared the binaries
  • Created the new mysql user, database again and sourced the sql

What actually need to be performed is

 --> Purge old, unneeded binary logs

Steps to perform

1) Type in console : 
 mysql> SHOW SLAVE STATUS \G;

This would show an output like follows :

************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 10.0.132.10
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000074
          Read_Master_Log_Pos: 1003370983
                        etc.

 2) If same as here if Slave_IO_state is still in reading state,it is safe to remove any logs created prior to the oldest one being read by replication slaves.

On the Replication master :


mysql> PURGE BINARY LOGS TO 'mysql-bin.000074';

Alternatively, logs prior to a certain date can be removed:

mysql> PURGE BINARY LOGS BEFORE '2011-01-01 00:00:00';

Note
This only takes care of the immediate excess of binary logs.
It does not prevent them from accumulating and resulting in low disk space again. To prevent this, the mysql global variable "expire log days" should be set. This variable determines how many days the binary logs are kept before being automatically purged.
By default it will be set to "0"

3)set expire log days

mysql> SET GLOBAL VARIABLE expire_logs_days=10;

 And also set mysql conf file : /etc/my.cnf
 


[mysqld] 
expire_logs_days = 10

Setting this variable will cause MySQL to automatically purge binary logs older than the number of days specified in the variable.


No we can check  the list of binary logs maintained by mysql by running


mysql> SHOW BINARY LOGS;
 
If the log files are not listed here,then successfully purged.

Reference -->
http://dev.mysql.com/doc/refman/5.0/en/purge-binary-logs.html


Thursday, June 19, 2014

Out of memory Issue in Tomcat

Description of Exception 
(Tomcat – java.lang.OutOfMemoryError: PermGen space)

The exception happens in web application deployment in tomcat.
Default settings memory space will not be adequate for some search information requirements.















If we check the tomcat error log, we might caught-up the issue as follows :
















Solution for Linux environment

1) Find out catalina.sh file in tomcat bin folder
           
2) Check for class path settings.
    Use setenv.sh instead to run the environment variable
    Set as follows :
   CLASSPATH=
 
    if [ -r "$CATALINA_BASE/bin/setenv.sh" ]; then
     . "$CATALINA_BASE/bin/setenv.sh"
    elif [ -r "$CATALINA_HOME/bin/setenv.sh" ]; then
     . "$CATALINA_HOME/bin/setenv.sh"
    fi
 
3) Check for setenv.sh in tomcat bin folder
    Check if it is there, if not create manually and enter the following to it.

export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xms128m -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=256m"

4) Restart Tomcat
   
      

 




Saturday, June 14, 2014

Generating keytool  for SSL configuration in Linux Command 


  • Navigate to the directory where we need to save your keystore file
  • Then type the following commands in the shell
  1. keytool -genkey -alias <name> -keyalg RSA -keypass changeit -storepass changeit -keystore keystore.jks
    (Note : Here name would be "tomcat" as default for the first time use.. If we need to create more than one keystore, then can use any name. If so, the name should be mentioned in tomcat server.xml)
  2. keytool -export -alias <name> -storepass changeit -file server.cer -keystore keystore.jks
  3. keytool -import -v -trustcacerts -alias <name> -file server.cer -keystore <path-to-jdk>/jre/lib/security/cacerts -keypass changeit -storepass changeit

  •  Then change the tomcat "server.xml" accordingly
    <-- Define a SSL Coyote HTTP/1.1 Connector on port 8443 -->
    <!--
    <Connector 
               port="8443" maxThreads="200"
               scheme="https" secure="true" SSLEnabled="true"
               keystoreFile="<keystore saved path>.keystore" keystorePass="changeit"
               clientAuth="false" sslProtocol="TLS"/>
    -->
       
  (Note : If you are using different port for https access then change port accordingly in server.xml) 
  •  Now we can access the deployed application securely. 
  • Go to the browser and type
    https://localhost:<configured-port-number> ie: here it is 8443

Saturday, June 7, 2014

Testopia and Bugzilla

Accessing Testopia

Any Bugzilla user can access Testopia by clicking on the "Product Dashboard" inside the main Bugzilla page


Products


The products in Testopia are imported from Bugzilla and are basically the products of the Yocto Project. These products can be seen in the left column of Testopia main page:




By clicking on a product you will see detailed description about it like the Test Plans it contains, the test cases and so forth.

Test Plans

  • test plans are unique to the product they belong to and can not be shared between products.
  • test plans contain all the test cases relevant to the product they belong to.


Test Cases

  • test cases can not be shared between products.
  • test cases can be shared between test plans within the same product.
  • test cases have 2 special fields assigned to them: priority and category.
  • test case priorities are linked to Bugzilla and affect bugs filed from the test case.

Test Runs

Test runs hold the results of how certain test cases fared in a specific build and environment. They are composed of Case Runs.
  • test runs are unique to a specific product.
  • test runs can not be shared between products.
  • test runs are unique to the test plan they belong to.

Bug Life Cycle States:

  • New - Potential defect that is raised and yet to be validated.
  • Assigned - Assigned against a development team to address it but not yet resolved.
  • Verified - The Defect that is retested and the test has been verified by QA.
  • Closed - The final state of the defect that can be closed after the QA retesting or can be closed if the defect is duplicate or considered as NOT a defect.
  • Reopened - When the defect is NOT fixed, QA reopens/reactivates the defect.

Bug Severity stages: 


SeverityDefinition
blockerPrevents function from being used, no work-around, blocking progress on multiple fronts
criticalPrevents function from being used, no work-around
majorPrevents function from being used, but a work-around is possible
normalA problem making a function difficult to use but no special work-around is required
minorA problem not affecting the actual function, but the behavior is not natural
trivialA problem not affecting the actual function, a typo would be an example

Bug Priority levels:

  • Urgent: Must to be fixed before any other high, medium or low defect should be fixed. Must be fixed in the next build.
  • High: Must be fixed in any of the upcoming builds but should be included in the release.
  • Medium: should take precedence over low priority defects and may be fixed after the release / in the next release.
  • Low: Fixing can be deferred until all other priority defects are fixed. It may or may not be fixed at all.