GUI Automation Basics - Custom Objects

Definition
An object is created from a class. Every tool will support set of basic classes. These classes are called as standard classes. If any class is not a standard class, it is called as Custom Class. Any objects, which are developed by custom classes is called as Custom Objects.

Types of custom objects:
Case1: Objects which are similar to objects of standard classes.
Example: Third party command buttons,Rich textbox, tool bars and Outlook components.

Case2: Objects which are not not similar to any objects of standard classes.
Example: Microsoft Excel, Word

Case3: Objects which are not able to identify by the tool
Example: Advanced technologies, Paint and charts.

Solution
Case1:
For example, Assume that my AUT's button class is a custom class. If developers used native API to develop buttons, those buttons would have been standard objects/controls. In this case, Custom objects are serving similar purpose of certain standard class. Another example is Rich text box, in which multiple lines can be entered.

In this case, we need to map custom class as corresponding standard class. After setting this, tool will start identifying the object as standard control instead of custom object. It is called as Class Mapping. Most of the tools are having flexibility of setting class mapping through coding or wizards.

Case2:
For Example, Assume that your AUT is an add-on of MS Excel. In Excel, you can set one or more cells as any objects like listbox, command button, picture.

In this case, assume that tool is identifying as whole Excel grid portion as a single custom object. You need to write needed supporting functions by using keyboard simulations. You should organize all the functions in a easy maintainability way.

Few tools like winrunner and QTP are providing a flexibility. You can point out certain portion or area as one object and you can write a functions for them.

Case3:
In case of advanced technologies, You should get proper support from tool vendor. Otherwise you can automate by using co-ordinates. In this case, you should use same system resolution and monitor size.

GUI Automation Basics - Object Identification

Two years ago, I was conducting an interview for automation engineer to our team. One candidate has totally 6 years Testing experience and it includes 1 year silktest automation experience.

My question is "what do you meant by framework?"
She replied that frame will be taken by using windows declarations.

It seems that she used to run the scripts and not developed scripts. I have seen many testers are trying to add automation experience in their resume. But they are not having basic understanding properly.

I thought, understanding Object Identification is needed first. Then only testers can do scripting properly for dynamic applications and custom objects. Most of the GUI automation tools are identifying the objects by following things. I have listed by priority.

Class - Tool is identifying the object's class. The class name would have defined by developer or used from generic controls.
Type of Object - Based on class, tool makes sure the type of object.
Title/Caption - The caption or title of the object. Windows, dialogs, menus and push button have caption.
Attached Text - The text which is used to identify the object. Example: TextField,CheckBox, Radio button, Combobox
Prior Text - The text which is prior to that particular object. It is different from attached text. This is identified by Left to Right or Right to Left order. Mostly this text is taken as Top to bottom.
Window ID - It is useful for stand alone applications. Windows OS assigns a ID for each object in a window.
Index - It depends upon the tool. Mostly tools are identifying based on similar objects.
Location - Identifying object by X & Y Co-ordinates.
Properties - Sometimes all above are not useful to identify the object uniquely. At that time, user should try that object properties.

Automation White Papers

I used to search framework papers, While starting automation in my first company. I have collected few links and docs. Some of them are not valid today. Listed favorite links in SQA Forums.

My Favorite papers
1. Totally Data-Driven Automated Testing by Keith Zambelich

2. Test Automation Frameworks by Carl J. Nagle

3. Architectures of Test Automation by Cem Kaner

4. Test Automation Snake Oil by James Bach

5. Success with Test Automation by Bret Pettichord

6. Seven Steps to Test Automation Success by Bret Pettichord

7. "When Should a Test Be Automated?" (Brian Marick)

8. Choosing a test automation framework from IBM

Regression Testing

Regression Testing:It retests previously tested segments to ensure that they still function properly after a change has been made to another part of the application.

Objectives:

• It determines whether systems documentation remains current
• It determines that system test data and test conditions remain current.

When do use regression testing:
When there is a high risk that new changes may affect unchanged areas of the application system.

Performance testing:
In this testing that the system will perform as specified at predetermined levels,wait times,static processes,dynamic processes and transaction processes.It is also tested at the client/browser and server levels.

Example:

• In database systems the response time relates to the time to obtain a report after clicking on a specific button.It may be difficult to specify the response times for each and every form/Report,but the time that can be generally specified,for example 30 seconds is reasonable.
• In real time or embedded systems the performance parameters are very significant.If the system demands,if the temperature exceeds 40 degrees then the valve will be opened with in 10milliseconds.Here this performance requirement is not met it may result in a catastrophe.

Hence performance testing is compulsory in process control and telecommunication software systems.

Sanity Testing:

Sanity testing is a cursory testing; it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing. It normally includes a set of core tests of basic GUI functionality to demonstrate connectivity to the database, application servers, printers, etc.

Smoke Testing:

Smoke testing is non-exhaustive software testing, ascertaining that the most crucial functions of a program work, but not bothering with finer details.

Usability testing

Nowadays we heard more about Usability Testing. It is getting popular for 3-4 years. It is so much important due to evolving new web technologies. We also planned to do Usability testing in our web based application. Mostly we targeted the UI consistency, navigations, objects alignment and colors.

Definition
Learning
Samples

I read few articles about it and did testing. Another worst part is, your development and product engineering team should have more interest on that. Most of the usability bugs are considered as low priority if functional testers raised. Few of them marked as high priority after clients reported. Here tester's pride is getting loss.

I have a question for you. Try to answer for following question.

Application that has GUI must be tested for usability - Choose the right answer.
a) For any kind of UI applications.
b) More necessary for web-based applications
c) Only if the user is going to use the UI.
d) Only if the user pays for it.

Expect - Automating Command line tools

Automating command line (CLI) tasks are always challenging for GUI automation tools. In my previous company, I used to use EXPECT for this kind of tasks. You can use telnet utility or Expect scripts to do this. I used EXPECT with Silktest. Expect is having a feature, you can feed the input based on the previous command's result. You can take baseline and compare with the current result.

You can connect with different operating Systems by expect scripts. It is developed as a package with Perl and Tcl/Tk. Three commands are central to the power of Expect. They are send, expect and spawn . The send command sends strings to a process, the expect command waits for strings from a process, and the spawn command starts a process.

Important Links
EXPECT home page
Windows version of EXPECT is available here
Getting Started

Sample
Call your EXPECT script by an batch or shell script. There you can pass the data as arguments.

4test statement:
SYS_Execute ("{GetProgramDir ()}\runftp_twiceexp.bat
{GetProgramDir ()}\ {lsInput[1]} {lsInput[2]}}} {lsInput[8]}", lsOutput)

Sample EXPECT code:

-----------------------------------------
##!/usr/local/bin/expect -f
#
# Automation for Unix system
#
#To delete the files from outbox table.
#
# hub name, installation path name, mailbox name is required.
#
# Author: Palani Selvam
#

# Check expect version
exp_version -exit 5.0.3

# root user login details
set username "root"
set upass "myrootpwd"

#super user login details
#set superuser "su"
#set supass "g0india"

#client,mailbox details
set pesetup_path "/opt/comp/pal_pnr"
set peclient "testhub"
set pmailbox "test_auto"

set timeout 10

#Start Telnet session
#spawn telnet jaguar

spawn telnet
expect "telnet>"
#send "telnet\r"
send "open jaguar\r"
sleep 1

expect {
  "ogin*" { 
 # In case login is "Login or login"
 send "$username\r"
 sleep 1
 }
  timeout {
 send_user "connection has timed out.\r\n"
 exit
 }
   }
expect {
  "assword*" {
  # In case password is "Password or password"
 send "$upass\r"
 sleep 5
 }
  timeout {
 send_user "password field has timed out.\r\n"
 exit
 }
  }

# Partner or hub
expect {
  "#*" {
 send "cd $pesetup_path/data/stpush\r"
 sleep 5
 send "cd $peclient/$pmailbox/outbox\r"
 sleep 4
 send "ls -ltr *\r"
 sleep 5
 }
  timeout {
 send_user "Super user login has failed.\r\n"
 exit
 }
  }

expect {
     "rw*" {
 send "cd completed\r"
 sleep 2
 send "rm *\r"
 sleep 2

 send "cd ../failed\r"
 sleep 2
 send "rm *\r"
 sleep 2

 send "cd ../partial\r"
 sleep 2
 send "rm *\r"
 sleep 2

 send "cd ../pending\r"
 sleep 2
 send "rm *\r"
 sleep 2

 send "cd ../retry\r"
 sleep 2
 send "rm *\r"
 sleep 2
 #puts "All files are deleted in outbox table.\r\n"
  }
  timeout {
 send_user "Files are not existing in outbox table.\r\n"
 exit
 }
   }

#To exit from the telnet
send "exit\r"
sleep 2
send "exit\r"

#Close the session
expect eof
-----------------------------------------

Phases in SDLC

Five different phases in software development lifecycle:

Each phase has a defined input and a defined output.

• Requirement analysis
• Design analysis
• Implementation
• Testing
• Maintenance

PHASES	INPUT	OUTPUT
Requirement Analysis	Get the requirements from client and problem definition.	Get the details of functionality and nonfunctionality requirements and Software requirement specifications (SRS)has to be written.
Design Analysis	Discuss about software requirement specifications(SRS) and analyze development of the architecture and design of the algorithms	It gives validated design document and details of various modules and the details of logic and algorithms
Implementation	Validated design document.software architects is given to programmer.They are able to do the coding.	Source code.Code in a given programming language.
Testing	The source code is converted into an executable code and various inputs are given.When wrong output is reflected defect is analyzed.	Completetely tested software that means Bug free software wil get.
Maintenance	It involves configuration management of the work product keeping track of the feedback from the client.	Customer satifaction

LinkTester - An Overview

In DotCom boom, Link testing is much popular. At that time, many tools and utilities are developed for link testing. It is mainly used to verify the site links in all pages. Now mostly used for documentation testing. Silktest is the only tool, contains link testing utility. Link Tester is a feature in SilkTest.

Link Tester is used to identify problems with hyperlinks in web sites/pages. It tests both standard hyperlinks, which link pages or an image to a URL, and visible links, which are links that exist on a web page but are not displayed in a certain context, for example image rollovers.

Linktester can be used only for HTTP URLs, It will not work for HTTPS(SSL) urls. I donno the limit of depth. But documentation recommends depth for 2. You can do by coding or accessing Tools->LinkTester.

Automation Tool support

In my first job, I used to automate by using VisualTest. There is no updates, after IBM acquired Rational. In this year, the yahoogroups also removed. VisualTest is a simple and powerful tool. Similarly Segue was acquired by Borland. Initially Borland took much time to set the updates on Silktest and Silkperformer. Later it announced that older versions of Silktest and silkperformer will not have support. Now its time for HP-Mercury.

Yesterday I got a mail from my friend. It said that HP is going to stop support for Winrunner. PDF is available here. There is a solution to migrate winrunner scripts to QuickTestProfessional. It is available at win-quick I donot know how much painful task it is for each company.


The frequent acquisitions are making many questions among to automation engineers.

1. Whether this product will be promoted?
2. Whether the updates will be added frequently as per changing technology?
3. What will happen to future if the product is ended?
4. How will be job market for this particular tool?

Software Testing Estimation

Estimation on Testing effort is always an interesting tophic. Testing Estimation depends upon the development estimation. Mostly I didn't see any separate estimation for testing (test execution activity). Always it is used to be assumed that testing should be completed within 3-4 builds once the feature is fully implemented. I know people are following different estimation techniques. All are based on their experience (see blog - pagkils).

Few of the known models are,
1. COCOMO Model
2. FP (Function Point) Analysis
3. WBS (Work Break-down Structure)

Last year, I had a different situation with my previous manager. Our application is a web based project and always the changes will be in existing functionalities. We had a tough time for Automation also. One day he said that fresher should execute 40-50 testcases per day. QA Engineer (whoever having 1-3 years experience) should execute twice the amount of a fresher. Senior QA Engineer (whoever having 4-6 years experience) should execute twice the amount of a QA engineer. None of them agreed this except freshers. Do you agree? Any comments?

Testing levels

Levels of Testing:
The whole software can not be tested at a time,so a practical approach is to divide the testing process into different levels.To start with each unit is tested separately and then the modules have to be built from the units and the modules are tested.Then the modules are combined together and the system is built and tested.

Level1:
Unit Testing:This testing is done to test the source code.Unit is the smallest piece of code that can be tested independently.This testing is done by the development team.After this testing units are combined into modules and then modules testing is carried out.

Level2:
Integration testing:This testing is done to test the design.Testing is carried out while combining different modules of a software package.Each module is tested separately and then modules are integrated and testing is carried out.In this testing debugging is easier by incrementally building the software.

Level3:
System testing:Testing is done to test the SRS(Software Requirement Specifications).After all the modules are combined together the system testing is carried out.Functional and performance testing are done during system testing.

Level4:
Acceptance Testing:Testing carriedout by the client to accept the software.This test procedure is prepared by the QA team inassociation with the client and testing is carried out as per this procedure.

Tip2 - CommandLine Tab

In unix systems, we will get a folder, which is under in current directory. Similarly we can make command prompt in Windows.

To do that, Go to “HKEY_CURRENT_USER\Software\Microsoft\Command Processor” in the registry (regedit.exe) and set the ‘CompletionChar’ value data as ‘9’. Here 9 refers the TAB key. Now close the command prompt and open a new command prompt. Now, if you press TAB key, the file/directory name will be displayed!.

Note: While using regedit, take care. Otherwise your system may not reboot properly...

Experts advise on Testing & General Software

Srini is a great tester in testing world.

Please read his advice for testing people.

Here is my advice to all aspiring QTP or Test engineers and professionals. These are lessons I learnt personally and useful for any software professional who is serious in testing.

1. Do not look for short cuts to learn and get knowledge. Have a long term plans to get good mileage in this profession. FAQs, etc are good to read only for knowing top line. To succeed in the interview you will have to win it from inside of your heart, invest honestly in studying and expect fruits. Banking on FAQs, interview questions etc may get you the job but will not keep you there.

2. Most important for a tester is to understand what makes a good tester? How he/she is different from a developer? What value tester brings to the table? How to find talent in testing and nurture it? How testing is different from QA or any flavor of process (CMM, Six sigma) etc.

3. Invest in sharpen problem solving and "thinking out of box" abilities. Read good stuff on testing. Participate in conferences, discuss with test professionals in other companies, participate in activities in SPIN, etc. Solve puzzles ( zig saw or shankuntala devi). Never
stop learning.

4. Sharpen Technology skills. Know about "How web works" , DNS, Networking, protocols, XML, Web services, Cryptography, databases, Datawarehousing, UNIX commands, Fundas of J2EE, .NET, system admin list is endless. Today testers are expected to know the basics. I take lot of interviews for various positions. Most of the people do not have these basics. It is difficult to survive in this world of testing only banking on "Automation tool" knowledge.

5. Lean programming languages like , C#, Java and scripting languages like PERL, python, Unix shell etc. This will increase utility value of yours. Developers and PMs will respect you.

6. Improve communication skills - take English class. Improve vocabulary. Read Read and Read. Most of the people I have seen ignore this important skill. They can not write a paragraph on their own without spelling and grammatical mistakes. Make a
habbit to learn a new word a day.

7. Read and write on blogs ( Google to find out what is blog- if you don't know already). Here are few blogs that I suggest for every test professional.

http://www.testingeducation.org/ - cem kaners free
testing courses.
http://www.testing.com/cgi-bin/blog - Brian Merick
site
http://blackbox.cs.fit.edu/blog/james/ - James Bach
- highly respected Visionary in testing.
http://www.qualitytree.com/index.html
http://blogs.msdn.com/micahel/ - Microsoft's Famous
Braidy Tester
http://www.developsense.com/blog.html - Michael
Bolten - Testing in plain English
http://www.stickyminds.com
http://www.kohl.ca/blog/ - Jonathan Kohl
http://www.io.com/~wazmo/blog/ - Bret Pettichord -
Automation testing Guru and

my own blogs -
http://blogs.msdn.com/shrinik ( my Microsoft blog -
is closed since left that company)
http://shrinik.blogspot.com

Last but not the least, Be a person with positive outlook in the life. Believe in yourself other wise nobody else will believe you.

All the best. Let us build a next generation test professionals community and change the way world does testing today.

Security Testing - Webscarab tool

Webscarab is a freeware. By using webscarab, we can do security testing for CSS and SQL injection. The URL is available here -> http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project

WebScarab is a framework for analysing applications that communicate using the HTTP and HTTPS protocols. It is written in Java, and is thus portable to many platforms. WebScarab has several modes of operation, implemented by a number of plugins. In its most common usage, WebScarab operates as an intercepting proxy, allowing the operator to review and modify requests created by the browser before they are sent to the server, and to review and modify responses returned from the server before they are received by the browser. WebScarab is able to intercept both HTTP and HTTPS communication.

Penetration testing is a method of evaluating the security of a computer system or network by simulating an attack by a malicious user, known as a cracker. The process involves an active analysis of the system for any potential vulnerabilities that may result from poor or improper system configuration, known and/or unknown hardware or software flaws, or operational weaknesses in process or technical countermeasures.

My thoughts about Test Automation Framework

Nowadays most of the people are talking about the framework. But most of them are not clear about the framework concepts. Here I'm writing my view about the framework.

Automation Framework is a concept, similar to OOPs. Automation Framework is an advanced thought from Record&Playback or writing some functions and executing them.

Automation Framework is a discipline and should have proper design/architecture. You can understand and implement. Automation Framework should have some of following components.

Components of a Framework
1. AUT Specific libraries
2. Tool wrapper function libraries.
3. Execution Engine (Script Execution should be drived
based on given test data) - Data Driven 4. Results
reporting mechanisms (PASS & FAIL for each
testcases/procedures and compiled results for all. It
would be better, if it captured AUT's snapshots) 5.
Planning for long term automation.
6. Keeping GUI objects info/declarations dynamically
or static.
7. Planned approach of unattended execution.

Also framework design differs by tool, scripting languages and type of AUT etc,. Now a days, people are using PERL, PYTHON, TCL/TK for CLI (Command Line Interface) automation. This design will vary much from GUI specific tool automation's design (QTP, Winrunner, Silktest,Robot and QARun etc).

You can go through following links. Each link is giving different set of components for Automation framework.
http://www.sqa-test.com/w_paper1.html
http://safsdev.sourceforge.net/FRAMESDataDrivenTestAutomationFrameworks.htm
http://www.qacity.com/NR/QACity/Documents/Automation_Framework.PDF
http://www.softwaredioxide.com/testing2004/papers/pankaj_cognizant.pdf
http://www.cbueche.de/FRM_DOC/webhelp_eng/Basic_Concepts.htm

I have developed different frameworks for Rational Visual Test, Winrunner and SilkTest. Some free frameworks are available freely on the net. EMOS is available for Winrunner and QTP. SAFS(http://safsdev.sourceforge.net) gives support for Winrunner and Robot. STAF(http://staf.sourceforge.net) will differ from SAFS.

Tip1 - To measure the App or System performance

Using Windows Performance monitor, we can collct performance data automatically from local or remote computers. Below steps will give you, how to set the counter in your machine or server. Depends upon the options, user can set any type of counter.

Steps:
1. GoTo Start->Run
2. Type "PerfMon" and Click OK button.
3. It will open performance app window.
4. GoTO Performance Logs and Alerts-> Counter Logs.
5. In the Right side Pane, right click on empty space and select 'New Log Settings'.
6. Enter the name for the log file.
7. In General Tab, you can see log file name. Click Add counter button.
8. Select that computer, 'Processor' as performance Object, '%Proccessor Time' as selected counter from list.
9. Also 'Total' as selected instance from list and click Add button.
10. Again select 'Memory' as performance Object, 'Availabile MBytes' as selected counter from list.
11. click Add button and close button. Set interval period.
12. Goto LogFiles TAB. Give log files path. Select 'Text File - CSV' as your log type and click apply button.
13. GoTo Schedule TAB and select manually.
14. Now setting entry has been created. To Run, Select that entry and Right click and select Start, before running your application.

Software Testing

Testing:
It is nothing but to detect the defect.It involves operation of a system or application under controlled conditions and evaluating the results.

Test is difficult because:

• We need to test both valid and invalid inputs and check the functionality as well as the performance parameters.
  
• We need to give the inputs randomly and checks that the software never fails.
• we need to test the software as though a normal user is using it and check whether the necessary error messages,help..etc are provided.
• We need to test the software by simulating the actual Environment.For example,if a database applicaion has to be accessed by 100 users simultaneously,it is not enough if you test the software for 2 or 10 users and declare that the software is working fine.
  

To summarize,testing is difficult,To carry out the testing in alimited time with limited effort and resources,and try to bring out all the possible hidden defects is a challenging and intelligent tasks.

Types of Testing:

1.Block Box Testing
2.White Box Testing

Software Engineering Code of Ethics and Professional Practice

Let me start with Ethics.Source: ACM org

Software engineers shall commit themselves to making the analysis, specification, design, development, testing and maintenance of software a beneficial and respected profession. In accordance with their commitment to the health, safety and welfare of the public, software engineers shall adhere to the following Eight Principles:

1. PUBLIC - Software engineers shall act consistently with the public interest.

2. CLIENT AND EMPLOYER - Software engineers shall act in a manner that is in the best interests of their client and employer consistent with the public interest.

3. PRODUCT - Software engineers shall ensure that their products and related modifications meet the highest professional standards possible.

4. JUDGMENT - Software engineers shall maintain integrity and independence in their professional judgment.

5. MANAGEMENT - Software engineering managers and leaders shall subscribe to and promote an ethical approach to the management of software development and maintenance.

6. PROFESSION - Software engineers shall advance the integrity and reputation of the profession consistent with the public interest.

7. COLLEAGUES - Software engineers shall be fair to and supportive of their colleagues.

8. SELF - Software engineers shall participate in lifelong learning regarding the practice of their profession and shall promote an ethical approach to the practice of the profession.

My first post

I'm going to write software testing tips and concepts. It will have details about automation testing and performance testing also. Have a regular look at my blog.