Extract info from Logs

One of my projects used to update set of log files for each action and scheduled actions. Exceptions are logged in those log files. Log file size is maintained upto 5 MB. We are unable to get the proper logs, if scripts are running more than 15 minutes. Entire suite runs almost 75 hours continuously.

Many times, we were unable to re-look at the exceptions in the logs for particular test case failed time. I thought to develop a vbscript to capture the exceptions from log file for frequent intervals and then write into another text file. It can be used for manual testing too. Here I made two things. First is, script has to parse the log file for the given string. Second is, script should not update the log information, which is already available. I meant, the same information should not be added multiple times.

Code to take only unique info It returns the information as Array. Array size is determined in run-time.

 
'-------------------------------------------------------------------------
' Method    :   TrimArrayToExtract
' Author    :   T. Palani Selvam
' Purpose   :   Remove the array elements based on given info (sItemToCheck).
' Parameters:   arrInput - Array String, Contains logging message.
'  sItemToCheck - String, to check the element match. Last element in the text file
' Returns   :   String - Array. Returns remaining elements from the given array.
' Caller    :   - Nil
' Calls     :   - Nil
'-------------------------------------------------------------------------
Function TrimArrayToExtract (arrInput, sItemToCheck)
    Dim iArrItem
    Dim sTemp
    Dim arrOutput()
    
    iIndex=0
    If (IsNull(sItemToCheck) Or IsEmpty(sItemToCheck) Or (Trim(sItemToCheck)= "")) Then
     TrimArrayToExtract=arrInput
     Exit Function
    Else    
     If (IsArray(arrInput)) Then
        'If (Not (IsNull(arrInput) Or IsEmpty(arrInput)) And (UBound(arrInput)> 1)) Then
        If (Not (IsNull(arrInput) Or IsEmpty(arrInput) Or (UBound(arrInput)=0)) ) Then
         'WScript.Echo "ArrInput : " & arrInput
      For iArrItem=0 to UBound(arrInput)  ''-1
   ReDim Preserve arrOutput(iIndex)
   sTemp=arrInput(iArrItem)
   If (sTemp=sItemToCheck) Then
      iIndex=0
      Erase arrOutput
   Else
      arrOutput(iIndex)=sTemp
      iIndex=iIndex+1
   End If

      Next 
    End If
  End If
   End If
    
   TrimArrayToExtract=arrOutput
   
End Function

Code for Extracting the information from Log file It returns the information in Array.

 
'--------------------------------------
' Method    :   ExtractInfoFromLog
' Author    :   T. Palani Selvam
' Purpose   :   To extract the lines from a log file based on given info.
' Parameters:   sFileName - String, contains the log filename with full path.
'               sInfo2Extract - String, Info to search.
' Returns   :   Array of String
' Caller    :   - Nil
' Calls     :   - Nil
'--------------------------------------
Function ExtractInfoFromLog (sFileName, sInfo2Extract)

 Dim objFSO, objTextFile, sLine
 Dim iPos
 Dim arrExtract()
 Dim iLinesToDo, iArrIndex, iLimit
 
 Const ForReading=1
 Const NoOfLines=10

 
 iArrIndex=0
 iLinesToDo=0
 iLimit=1
 
 Set objFSO = CreateObject("Scripting.FileSystemObject")
 If  objFSO.FileExists(sFileName) Then

   Set objTextFile = objFSO.OpenTextFile(sFileName, ForReading) 

   Do while Not objTextFile.AtEndOfStream
  sLine = objTextFile.ReadLine        
  'Wscript.Echo sLine
  If (iLinesToDo > 0) Then
   arrExtract(iArrIndex)=sLine   
   iLinesToDo=iLinesToDo-1
   iArrIndex=iArrIndex+1
  Else
   
   iLinesToDo=0
   iPos=InStr(sLine,sInfo2Extract)
   'WScript.Echo("iPos  " & CStr(iPos))
   If (iPos>0) Then
    'WScript.Echo("Entered into If loop to extract from line: " & sLine)

    '' iLimit = UBound (arrExtract) + NoOfLines
    iLimit = iLimit + NoOfLines
    ReDim Preserve arrExtract(iLimit)

    arrExtract(iArrIndex)=sLine   
    iLinesToDo=NoOfLines-1 
    iArrIndex=iArrIndex+1

   Else
    iLinesToDo=0

   End If
  End If
  
   Loop

 End If

 objTextFile.Close

 Set objTextFile = Nothing 
 Set objFSO = Nothing 
 
 ExtractInfoFromLog=arrExtract
End Function

Note: Few user-defined functions might be used.

Is KeyWord Driven Framework enough?

Recently I heard the term Key-Word Driven Framework from many testing people. Even people are showing interest to know about it and to implement it. Sometimes people are saying unknowingly as Keyboard Driven Framework.

Few months back, I have implemented Keyword Driven framework in one of our projects. We are using Silktest and data files are kept in XML format. The other automation team members also impressed with this kind of implementation. I found this article Automated Testing Institute - Building A Keyword Driver Script In 4 Steps useful to the professionals, who wants to implement KeyWord Driven framework.

After that the expectation goes to implement KeyWord Driven Framework to all automation projects. I am not against to Keyword Driven Framework. It can be used for many projects. But it is not the only one solution for all our automation issues or dependencies. For example, chart automation. For charts, verification will be different and It is difficult to generalize. May be Flex charts can be used if tools are able to identify all parts of charts. One more is unlimited Keywords. Assume that one project contains more than 500 keywords and the automation guys should know the functionality for all those words. It is definitely issue, if number of keywords increased beyond the limit. Currently I'm thinking the scenarios, where KeyWord or Table Driven Automation frameworks can not be used or should be avoided. Below I have listed the cases.

1. More Context based Menus
   The menu items are shown based on the screen and object.


2. Multiple Application interactions in single project


3. Total Keywords beyond 200
   Testers should memorize all keywords. It is similar to remember Differential Calculus and Integral Calculus functions. One should remember keywords for action and verification for all set of objects.


4. Many hierarchies for Object identification.
   It can be solved if testing tool supports X-path for object identification.


5. Complex data.
   One of my projects required minimum 40 string data to create a report and it is just a first step for any test case. Also the data may change in the future. More than 1000 cases required like that. I can generalize the inputs data and it is difficult to maintain and modify the data.

To know the automation framework concepts, you can go through following links.

Test Automation White Papers
My Thoughts about Automation
What is an Automation Framework - from Dhanasekar's blog
Wikipedia - Test Automation
Wikipedia - KeyWord Driven Testing

PDF File Verification

Sometimes testers need to verify the PDF file contents. Few times I have seen the questions related to this in few forums. Verifying graphical contents are not so easy. But we can verify the text content in three ways.

First way - Scripting
You can use any other scripting to verify PDF. You can use Java Script or VB Script. In this way, PDF file will not be opened physically and retrieve the contents internally. For more info, Read through this link - Accessing PDF

Second way - Utility
Convert the PDF files to text by using any utility and then verify text files. There are many freewares available for this kind of purpose. I suggest TextMining Tool. In this way also, PDF file is not opened physically.

Sample code in Silktest

 
STRING sPDF2TxtUtil = "F:\TextMining\minetext.exe"
sCmdExecute = "{sPDF2TxtUtil} {sPdfFile} {sConvertedFile}"
Print ("Command: {sCmdExecute}")
SYS_Execute (sCmdExecute,lsCmdOut)
Print (lsCmdOut)

Third way - Using Adobe Reader
In this way, Open the pdf file using Adobe Reader. Then go to File Menu and then click sub menu Save As Text. Now you can store pdf contents as text file and you can use the text file for verification.

To compare the PDF files, you can use comparison softwares (for ex Beyond Compare from 3.0). They internally convert to text file and then comparing it.

Java and myself

In my first job, I have used VB and Java. At that time, I was new to Internet. I used to develop programs in VB6 and Java2 around Internet programming. I wrote java programs using notepad and Emacs editors. After that many years I ran behind GUI automation tools such as VisualTest, WinRunner, QTP and Silktest. Sometimes I'm using VBScript for few testing tasks.

Recently many tools are supported to use Java as test language. Again I'm going to re-look at java. I was looking into my old java code. I tried to compile few java programs with recent Java version 1.6.0.14. Few classes are deprecated. I have used this command javac ZDownloadURL.java -Xlint:deprecation to compile java program.

In Sun Developer network, one tutorial was there for Basic Java. Link :Essentials of the Java Programming Language

Below program can be used to download any web page. I tried my blog and it still works.
To Compile - javac ZDownloadURL.java -Xlint:deprecation
To run the program - java ZDownloadURL

 
// $Id$

/**
 *
 *
 * Created: Tue Oct 24 18:51:26 2000
 *
 * @author palani selvam
 * @version $Revision$
 */

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.net.*;
import java.io.*;


/*  To Run this file, put command as
java -Dhttp.proxyHost=192.168.27.2 -Dhttp.proxyPort=3128 ZDownloadURL
*/


public class ZDownloadURL extends JFrame implements WindowListener,ActionListener {
    String URL;    //Given URL
    //    String inputLine; //To read Line by line
    String title="Downloading Trial";     //To give title of the URl

    static JFrame frame;       //Parent Level Container
    JButton save;       //Save Button
    JButton down;        //Declare Download Button
    JButton exit;        //exit Button
    JTextField url;      //To enter URl
    JTextArea contents;   //Contents of specific file
    //JPanel panel;         //Second level Container
    Container panel;
    //JPanel temp;
    Container temp;
    
    
    public ZDownloadURL() {
        //frame=new JFrame();
        super.setTitle(title);
        super.setForeground(Color.blue);
        super.setBackground(Color.yellow);
        
        panel=getContentPane();
        temp=new JPanel();
        temp.setLayout(new BorderLayout());
        temp.setSize(10,20);
        
        GridBagLayout grid=new GridBagLayout();                  //Set GridBagLayout
        GridBagConstraints c=new GridBagConstraints();           
        c.fill=GridBagConstraints.HORIZONTAL;
        
        panel.setLayout(grid);
        // grid.setConstraints(c);                                //Add GridbagConstraints

        url=new JTextField(20);
        contents=new JTextArea(30,30);
        contents.setLineWrap(true);                 //To Wrap lines in TextArea

/***
* To create ScrollBars to JTextArea
*
*/

 JScrollPane pane = new JScrollPane(contents);        
        temp.add(pane);



        /***
        contents.setColumns(20);
        contents.setRows(10);
        contents.setLineWrap(true);
        contents.setWrapStyleWord(true);
        contents.setSize(10,20);

 temp.add(contents);
        **/
        
        
        
        down=new JButton("DownLoad");
        save=new JButton("Save");
        exit=new JButton("Exit");
        JLabel label=new JLabel("Enter valid URL");

        c.gridx=0;
        c.gridy=0;
        c.insets=new Insets(5,0,0,0);
        grid.setConstraints(label,c);
        panel.add(label);

        c.gridx=1;
        c.gridy=0;
        c.gridwidth=1;
        //c.gridheight=2;
        c.insets=new Insets(5,0,0,0);
        grid.setConstraints(url,c);
        panel.add(url);

        c.gridx=2;
        c.gridy=0;
        c.insets=new Insets(5,0,0,0);
        grid.setConstraints(down,c);
        panel.add(down);

        c.gridx=0;
        c.gridy=1;
        c.gridwidth=20;
        c.gridheight=10;
        c.insets=new Insets( 20,0,0,0);
        grid.setConstraints(temp,c);
        panel.add(temp);
         

        c.gridx=0;
        c.gridy=12;
        c.gridwidth=1;
        c.insets=new Insets(10,0,0,0);
        grid.setConstraints(save,c);
        panel.add(save);

        c.gridx=1;
        c.gridy=12;
        c.gridwidth=1;
        c.insets=new Insets(10,0,0,0);
        grid.setConstraints(exit,c);
        panel.add(exit);

        exit.addActionListener(this);
        save.addActionListener(this);
        down.addActionListener(this);
        //        frame.addActionListener(this);
        
        //    frame.add(panel);
        
        
    }


    public void windowActivated(WindowEvent we) {

    }

    public void windowDeactivated(WindowEvent we) {

    }

    public void windowIconified(WindowEvent we) {

    }

    public void windowDeiconified(WindowEvent we) {

    }

    public void windowOpened(WindowEvent we) {

    }

    public void windowClosed(WindowEvent we) {

    }

    public void windowClosing(WindowEvent we) {

        System.exit(10);

    }


/***
* pass the URL as string
*/


    public void Download(String _url) {
        try {
        URL myURL=new URL(_url);
        BufferedReader in=new BufferedReader(new InputStreamReader(myURL.openStream()));

        String inputLine;
        contents.setText("");

        while((inputLine=in.readLine()) != null) {
            //contents.setText(contents.getText()+"\n"+inputLine);
            contents.append("\n"+inputLine);

        }
        } catch(Exception ex) {

            System.out.println(ex.toString());
        }

    }

    public boolean checkURL(String _url) {
        if((_url.indexOf("http"))!=0) {
            return true;
        }
        else {
            return false;
        }
    }
           
    
    public void actionPerformed(ActionEvent ae) {
        //String action=ae.getActionCommand();

        URL=url.getText();
        Object action=ae.getSource();
        
        //To Download

        if(action.equals(down)) {
            /**    if(checkURL(URL)) {
                Download(URL);
            }
            else
                JOptionPane.showMessageDialog(new JFrame(),"Enter Valid URL in TextField!","Information",JOptionPane.INFORMATION_MESSAGE);

            **/

            Download(URL);

                
           }


 //To save that URL's contents
 //Not implemented

        if(action.equals(save)) {
            JFileChooser fc=new JFileChooser();
            int returnVal=fc.showSaveDialog(new JFrame());

            if(returnVal==JFileChooser.APPROVE_OPTION) {
                File file=fc.getSelectedFile();
                super.setTitle(file.getName());
            }
        }

           if(action.equals(exit)) {
               System.exit(0);
           }
           
    }

           public static void main(String args[]) {
               frame=new ZDownloadURL();

               frame.setSize(500,400);
               frame.pack();
               frame.show();
    
           }
    
} // ZDownloadURL

/*
 * $Log$
 */