Tools learning for testing

Many times folks used to ask, what tools would be good for testing career. Used to advice to learn popular tools (mostly commercial tools) based on their coding skills and interest. But this scenario is changed in last few years. Testers should be ready to handle the automation based on business needs, rather than just toolsmiths. Also mindset change should be there, instead of sticking with just one tool and one scripting language

Read an article recently in this line and few excerpts..


It’s important to ask yourself, why do you want to learn about a specific tool or set of tools? They are an integral part of software testing, supporting what we do on a daily basis. However, having knowledge of a set of tools is less important than knowing when and where to use the right tool. An effective tester will also have the skill to know when and where to use the right tool because the problem requires it, not because the job market determines it desirable. Using the wrong tool at the wrong time can have adverse effects on your day-to-day work, potentially slowing you down, or worse, giving you biased or flat out false information. That’s not to say you shouldn’t explore new tools, but always remember:

Problem first, tools second
With this concept in mind, let’s explore different categories of tools. Some examples will be offered but they are by no means an exhaustive list. I encourage you to build up a toolbox or reference list of tools you have used in the past.

Original Article - What Tools Should I Learn?

Security Attacks - OWASP Top 10

I was looking into cross site scripting for security testing. Look at OWASP top 10 attacks for year 2010, 2007, 2004. Few items have gone up or down, added and removed.

Year 2010 - Top 10

1. Injection
2. Cross-Site Scripting (XSS)
3. Broken Authentication and Session Management
4. Insecure Direct Object References
5. Cross-Site Request Forgery (CSRF)
6. Security Misconfiguration
7. Insecure Cryptographic Storage
8. Failure to Restrict URL Access
9. Insufficient Transport Layer Protection
10. Unvalidated Redirects and Forwards

Year 2007 - Top 10

1. Cross Site Scripting (XSS)
2. Injection Flaws
3. Malicious File Execution
4. Insecure Direct Object Reference
5. Cross Site Request Forgery (CSRF)
6. Information Leakage and Improper Error Handling
7. Broken Authentication and Session Management
8. Insecure Cryptographic Storage
9. Insecure Communications
10. Failure to Restrict URL Access

Year 2004 - Top 10

1. Unvalidated Input
2. Broken Access Control
3. Broken Account and Session Management
4. Cross Site Scripting (XSS) Flaws
5. Buffer Overflows
6. Injection Flaws
7. Improper Error Handling
8. Insecure Storage
9. Denial of Service
10. Insecure configuration management

Database Testing

Today database becomes as engine to many of the enterprise applications. System would be affected badly if the transactions and queries produced the false conditions. Due to lack of database testing, applications may go to dead lock, data corruption and data loss conditions. These type of issues would take more time to identify and fix. Database testing includes verifying stored procedures, table indexes, exceptions, schemas and compatibility.

Different types of Database Testing

• Structural testing
• Functional testing
• Boundary testing
• Stress Testing

Few Sample Scenarios

• Creating an user account from GUI - How would you ensure the details are stored into table correctly?
• Executing stored procedures in different conditions like valid and invalid conditions.
• Varying data definitions - The data type and length for a particular attribute may vary in tables though the semantic definitions are same. Example: Account number declared as Number ( 9 ) in one table and the same as varchar2( 11 ) in another table.
• Varying data codes and values - The data representation of the same attribute may vary with and across tables. Example: Yes or No may be represented as "Y", "y", "N", "n", "1", "0".
• Misuse of integrity constraints - When referential integrity constrains are misused, foreign key values may left "dangling". Example: Employee record deleted but dependent records not deleted.
• Nulls - Null may be ignored when joining tables or doing searches on the column.
• Inaccessible data - Inaccessible data due to missing or reduntant unique identifier value. Example: Uniqueness not enforced.
• Incorrect data values - Data that is misspelled or inaccurately recorded. Example: Indra Nagar - Indra ngr.
• Inappropriate use of views - Data is updated incorrectly through views. Example: Data is properly fetched from the database but first record or last record is not displayed

Extract info from Logs

One of my projects used to update set of log files for each action and scheduled actions. Exceptions are logged in those log files. Log file size is maintained upto 5 MB. We are unable to get the proper logs, if scripts are running more than 15 minutes. Entire suite runs almost 75 hours continuously.

Many times, we were unable to re-look at the exceptions in the logs for particular test case failed time. I thought to develop a vbscript to capture the exceptions from log file for frequent intervals and then write into another text file. It can be used for manual testing too. Here I made two things. First is, script has to parse the log file for the given string. Second is, script should not update the log information, which is already available. I meant, the same information should not be added multiple times.

Code to take only unique info It returns the information as Array. Array size is determined in run-time.

 
'-------------------------------------------------------------------------
' Method    :   TrimArrayToExtract
' Author    :   T. Palani Selvam
' Purpose   :   Remove the array elements based on given info (sItemToCheck).
' Parameters:   arrInput - Array String, Contains logging message.
'  sItemToCheck - String, to check the element match. Last element in the text file
' Returns   :   String - Array. Returns remaining elements from the given array.
' Caller    :   - Nil
' Calls     :   - Nil
'-------------------------------------------------------------------------
Function TrimArrayToExtract (arrInput, sItemToCheck)
    Dim iArrItem
    Dim sTemp
    Dim arrOutput()
    
    iIndex=0
    If (IsNull(sItemToCheck) Or IsEmpty(sItemToCheck) Or (Trim(sItemToCheck)= "")) Then
     TrimArrayToExtract=arrInput
     Exit Function
    Else    
     If (IsArray(arrInput)) Then
        'If (Not (IsNull(arrInput) Or IsEmpty(arrInput)) And (UBound(arrInput)> 1)) Then
        If (Not (IsNull(arrInput) Or IsEmpty(arrInput) Or (UBound(arrInput)=0)) ) Then
         'WScript.Echo "ArrInput : " & arrInput
      For iArrItem=0 to UBound(arrInput)  ''-1
   ReDim Preserve arrOutput(iIndex)
   sTemp=arrInput(iArrItem)
   If (sTemp=sItemToCheck) Then
      iIndex=0
      Erase arrOutput
   Else
      arrOutput(iIndex)=sTemp
      iIndex=iIndex+1
   End If

      Next 
    End If
  End If
   End If
    
   TrimArrayToExtract=arrOutput
   
End Function

Code for Extracting the information from Log file It returns the information in Array.

 
'--------------------------------------
' Method    :   ExtractInfoFromLog
' Author    :   T. Palani Selvam
' Purpose   :   To extract the lines from a log file based on given info.
' Parameters:   sFileName - String, contains the log filename with full path.
'               sInfo2Extract - String, Info to search.
' Returns   :   Array of String
' Caller    :   - Nil
' Calls     :   - Nil
'--------------------------------------
Function ExtractInfoFromLog (sFileName, sInfo2Extract)

 Dim objFSO, objTextFile, sLine
 Dim iPos
 Dim arrExtract()
 Dim iLinesToDo, iArrIndex, iLimit
 
 Const ForReading=1
 Const NoOfLines=10

 
 iArrIndex=0
 iLinesToDo=0
 iLimit=1
 
 Set objFSO = CreateObject("Scripting.FileSystemObject")
 If  objFSO.FileExists(sFileName) Then

   Set objTextFile = objFSO.OpenTextFile(sFileName, ForReading) 

   Do while Not objTextFile.AtEndOfStream
  sLine = objTextFile.ReadLine        
  'Wscript.Echo sLine
  If (iLinesToDo > 0) Then
   arrExtract(iArrIndex)=sLine   
   iLinesToDo=iLinesToDo-1
   iArrIndex=iArrIndex+1
  Else
   
   iLinesToDo=0
   iPos=InStr(sLine,sInfo2Extract)
   'WScript.Echo("iPos  " & CStr(iPos))
   If (iPos>0) Then
    'WScript.Echo("Entered into If loop to extract from line: " & sLine)

    '' iLimit = UBound (arrExtract) + NoOfLines
    iLimit = iLimit + NoOfLines
    ReDim Preserve arrExtract(iLimit)

    arrExtract(iArrIndex)=sLine   
    iLinesToDo=NoOfLines-1 
    iArrIndex=iArrIndex+1

   Else
    iLinesToDo=0

   End If
  End If
  
   Loop

 End If

 objTextFile.Close

 Set objTextFile = Nothing 
 Set objFSO = Nothing 
 
 ExtractInfoFromLog=arrExtract
End Function

Note: Few user-defined functions might be used.

Difference between Application server and Web server

I have explained the difference between Application server and web server. I prepared this table after referring few sites. I hope that it will be very useful to you.

Application server	Web server
Developers can create, test, and execute application components.	It is designed to create and deploy Web site, serving up content more so than applications.
These are typically J2EE-based, running EJBs or other Java components.	It supports JSP,Servlets,ASP and server side Java Script.
It exposes business logic to client applications through various protocols.	It is designed to create and deploy Web site serving.It mainly deals with sending HTML for display in a web browser.
Application Server supports HTTP,TCP/IP and many more protocols.	A Web Server understands and supports only HTTP protocol.

Common Problems in Test Automation

Few years back, I have prepared few documents about automation scripting and concepts. This post will list the common problems faced by software industries in their test automation attempts.

Automation is not cheap
Test Automation is not cheap. Usually it takes between 3 to 10 times as long to create, verify and document the automated test as it takes to create and run once by hand. Test Automation takes lesser time to create than to test each test scripts.

Easy to run test alone is automated
Many test groups automate only the easy to run tests because early in testing, these are easy to design and the program might not be capable of running more complex test cases. Hence harsh test needs to be done by the skilled manual tester.

Duplicate information was kept in multiple repositories
Most of the project teams purchases a test management tool in addition to the already existing automated testing tools. Duplicate information was kept in multiple repositories and was very difficult to maintain. In several instances, the implementation of more tools actually results in less productivity.

Test Scripts duplicates the development effort
Test script development results in an almost complete duplication of the development effort, through overuse of the testing tool’s programming language. In normal case the application itself used a complex algorithm and again the tester recreates these algorithms using the testing tool. Too much time was spent on automating scripts, without much additional value gained. The test team must be careful not to duplicate the development effort; this is a risk when developing elaborate test scripts.

A lack of test development guidelines was noted
One program had several test automation engineers, each using a different style for creating test scripts. Maintaining the scripts is a nightmare. This is due to the non-availability of development guidelines and hence script readability and maintainability becomes complicated.

Reports generated by the tool is futile
More time is spent by the testers in setting up elaborate customized reports, which is part of the automated testing tool. The reports were never used, since the data required for the report is never accumulated in the tool.

Security Testing - CSS or XSS

The Cross Site Scripting (also known as XSS or CSS) is one of the most common Security Testing in web applications. Recently It was implemented in our application. We certified our AUT for CSS.

Generally hackers try to embed malicious script into a vulnerable dynamic web applications. This malicious script is executed and hacker can steal the data. The use of XSS might compromise private information, manipulate or steal cookies, create requests that can be mistaken for those of a valid user, or execute malicious code on the end-user systems.

To avoid cross site scripting vulnerabilities, the application should use encoded HTML content and it should not allow any URL or data, which contains <script and %3C%2Fscript.

In web so many tools are available to test this. I used a freeware called Webscarab. Also you can go through following links to know more about XSS.

Cross Site Scripting Definition - Wiki
Cross site Scripting FAQ
Sample Videos from Microsoft

Documentation testing

Recently I have done a review for installation and user guides. I came to know few basic issues. Document testing is very important for user guide and installation guides. It can bring few more values to customers. Generally testers are not interested to test the documents. You will change your mindset if you read David's article - Justification for Documentation Testing.

Documentation testing is nothing but testing concerned with the accuracy of documentation. Documentation meets its goal when it provides enough and necessary information to end users or customers. Below I have given a famous technical writer Richard Lippincott's comment.
Quote 1:

In recent years, usability has been recognized as an important
issue in documentation quality, and methodologies
have been developed to test and improve the user’s
speed, readability, and ability to find information.

Quote 2:

The documentation testing process can be considered the
test for the accuracy portion of the usability model.
It would fall under the key practice of quality assurance
activities as defined by Hackos in the Information
Process Maturity Model.

Testing Computer Software (ISBN: 0471358460) book talks about testing user manuals. It says that the effective documentation should have following benefits:

1. Improves Usability


2. Lowers customer support costs


3. Improves reliability


4. Increases Maintainability


5. Improves Installability


6. Enhances salability


7. Reduces liability

Web Testing & Client server testing

Server process:

 Server programs generally receive requests from client programs, execute database retrieval
and updates, manage data integrity and dispatch responses to client requests. The serverprocess acts as a software engine
that manages shared resources such as databases, printers, communication links, or high
powered-processors.It is the backend process of the application.

Client process: 
       Client programs usually manage the user-interface portion of theapplication, validate data
entered by the user, dispatch requests toserver programs, and sometimes execute business logic.
The client-based process is the front-end of the application.It is the interaction between the user and the rest of the application
system.

In client server testing test engineer are conduct the following testings:-

1.Behaviour testing(GUI TESTING)

2.Input domain testing

3.Error Handling testing

4.Backend testing

In Web testing test engineer are condut the following testings:-

1.Behaviour Testing

2.Static web testing

3.Input domain testing

4.Backend testing

5.Error handling testing

5.Frame Level testing

Difference between Application server and Web server:

Application server	Web server
Developers can create, test, and execute application components	It is designed to create and deploy Web site, serving up content more so than applications.
These are typically J2EE-based, running EJBs or other Java components.	It supports JSP,servlets and ASP
Application servers are designed to create true applications with complex business logic,	Web servers are technology designed to create and deploy Web site serving up content more so than applications, serving
Application Server supports HTTP,TCP/IP and many more protocols.	A Web Server understands and supports only HTTP protocol

Usability testing

Nowadays we heard more about Usability Testing. It is getting popular for 3-4 years. It is so much important due to evolving new web technologies. We also planned to do Usability testing in our web based application. Mostly we targeted the UI consistency, navigations, objects alignment and colors.

Definition
Learning
Samples

I read few articles about it and did testing. Another worst part is, your development and product engineering team should have more interest on that. Most of the usability bugs are considered as low priority if functional testers raised. Few of them marked as high priority after clients reported. Here tester's pride is getting loss.

I have a question for you. Try to answer for following question.

Application that has GUI must be tested for usability - Choose the right answer.
a) For any kind of UI applications.
b) More necessary for web-based applications
c) Only if the user is going to use the UI.
d) Only if the user pays for it.

Phases in SDLC

Five different phases in software development lifecycle:

Each phase has a defined input and a defined output.

• Requirement analysis
• Design analysis
• Implementation
• Testing
• Maintenance

PHASES	INPUT	OUTPUT
Requirement Analysis	Get the requirements from client and problem definition.	Get the details of functionality and nonfunctionality requirements and Software requirement specifications (SRS)has to be written.
Design Analysis	Discuss about software requirement specifications(SRS) and analyze development of the architecture and design of the algorithms	It gives validated design document and details of various modules and the details of logic and algorithms
Implementation	Validated design document.software architects is given to programmer.They are able to do the coding.	Source code.Code in a given programming language.
Testing	The source code is converted into an executable code and various inputs are given.When wrong output is reflected defect is analyzed.	Completetely tested software that means Bug free software wil get.
Maintenance	It involves configuration management of the work product keeping track of the feedback from the client.	Customer satifaction

Testing levels

Levels of Testing:
The whole software can not be tested at a time,so a practical approach is to divide the testing process into different levels.To start with each unit is tested separately and then the modules have to be built from the units and the modules are tested.Then the modules are combined together and the system is built and tested.

Level1:
Unit Testing:This testing is done to test the source code.Unit is the smallest piece of code that can be tested independently.This testing is done by the development team.After this testing units are combined into modules and then modules testing is carried out.

Level2:
Integration testing:This testing is done to test the design.Testing is carried out while combining different modules of a software package.Each module is tested separately and then modules are integrated and testing is carried out.In this testing debugging is easier by incrementally building the software.

Level3:
System testing:Testing is done to test the SRS(Software Requirement Specifications).After all the modules are combined together the system testing is carried out.Functional and performance testing are done during system testing.

Level4:
Acceptance Testing:Testing carriedout by the client to accept the software.This test procedure is prepared by the QA team inassociation with the client and testing is carried out as per this procedure.

Experts advise on Testing & General Software

Srini is a great tester in testing world.

Please read his advice for testing people.

Here is my advice to all aspiring QTP or Test engineers and professionals. These are lessons I learnt personally and useful for any software professional who is serious in testing.

1. Do not look for short cuts to learn and get knowledge. Have a long term plans to get good mileage in this profession. FAQs, etc are good to read only for knowing top line. To succeed in the interview you will have to win it from inside of your heart, invest honestly in studying and expect fruits. Banking on FAQs, interview questions etc may get you the job but will not keep you there.

2. Most important for a tester is to understand what makes a good tester? How he/she is different from a developer? What value tester brings to the table? How to find talent in testing and nurture it? How testing is different from QA or any flavor of process (CMM, Six sigma) etc.

3. Invest in sharpen problem solving and "thinking out of box" abilities. Read good stuff on testing. Participate in conferences, discuss with test professionals in other companies, participate in activities in SPIN, etc. Solve puzzles ( zig saw or shankuntala devi). Never
stop learning.

4. Sharpen Technology skills. Know about "How web works" , DNS, Networking, protocols, XML, Web services, Cryptography, databases, Datawarehousing, UNIX commands, Fundas of J2EE, .NET, system admin list is endless. Today testers are expected to know the basics. I take lot of interviews for various positions. Most of the people do not have these basics. It is difficult to survive in this world of testing only banking on "Automation tool" knowledge.

5. Lean programming languages like , C#, Java and scripting languages like PERL, python, Unix shell etc. This will increase utility value of yours. Developers and PMs will respect you.

6. Improve communication skills - take English class. Improve vocabulary. Read Read and Read. Most of the people I have seen ignore this important skill. They can not write a paragraph on their own without spelling and grammatical mistakes. Make a
habbit to learn a new word a day.

7. Read and write on blogs ( Google to find out what is blog- if you don't know already). Here are few blogs that I suggest for every test professional.

http://www.testingeducation.org/ - cem kaners free
testing courses.
http://www.testing.com/cgi-bin/blog - Brian Merick
site
http://blackbox.cs.fit.edu/blog/james/ - James Bach
- highly respected Visionary in testing.
http://www.qualitytree.com/index.html
http://blogs.msdn.com/micahel/ - Microsoft's Famous
Braidy Tester
http://www.developsense.com/blog.html - Michael
Bolten - Testing in plain English
http://www.stickyminds.com
http://www.kohl.ca/blog/ - Jonathan Kohl
http://www.io.com/~wazmo/blog/ - Bret Pettichord -
Automation testing Guru and

my own blogs -
http://blogs.msdn.com/shrinik ( my Microsoft blog -
is closed since left that company)
http://shrinik.blogspot.com

Last but not the least, Be a person with positive outlook in the life. Believe in yourself other wise nobody else will believe you.

All the best. Let us build a next generation test professionals community and change the way world does testing today.

Security Testing - Webscarab tool

Webscarab is a freeware. By using webscarab, we can do security testing for CSS and SQL injection. The URL is available here -> http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project

WebScarab is a framework for analysing applications that communicate using the HTTP and HTTPS protocols. It is written in Java, and is thus portable to many platforms. WebScarab has several modes of operation, implemented by a number of plugins. In its most common usage, WebScarab operates as an intercepting proxy, allowing the operator to review and modify requests created by the browser before they are sent to the server, and to review and modify responses returned from the server before they are received by the browser. WebScarab is able to intercept both HTTP and HTTPS communication.

Penetration testing is a method of evaluating the security of a computer system or network by simulating an attack by a malicious user, known as a cracker. The process involves an active analysis of the system for any potential vulnerabilities that may result from poor or improper system configuration, known and/or unknown hardware or software flaws, or operational weaknesses in process or technical countermeasures.

Tip1 - To measure the App or System performance

Using Windows Performance monitor, we can collct performance data automatically from local or remote computers. Below steps will give you, how to set the counter in your machine or server. Depends upon the options, user can set any type of counter.

Steps:
1. GoTo Start->Run
2. Type "PerfMon" and Click OK button.
3. It will open performance app window.
4. GoTO Performance Logs and Alerts-> Counter Logs.
5. In the Right side Pane, right click on empty space and select 'New Log Settings'.
6. Enter the name for the log file.
7. In General Tab, you can see log file name. Click Add counter button.
8. Select that computer, 'Processor' as performance Object, '%Proccessor Time' as selected counter from list.
9. Also 'Total' as selected instance from list and click Add button.
10. Again select 'Memory' as performance Object, 'Availabile MBytes' as selected counter from list.
11. click Add button and close button. Set interval period.
12. Goto LogFiles TAB. Give log files path. Select 'Text File - CSV' as your log type and click apply button.
13. GoTo Schedule TAB and select manually.
14. Now setting entry has been created. To Run, Select that entry and Right click and select Start, before running your application.

Software Testing

Testing:
It is nothing but to detect the defect.It involves operation of a system or application under controlled conditions and evaluating the results.

Test is difficult because:

• We need to test both valid and invalid inputs and check the functionality as well as the performance parameters.
  
• We need to give the inputs randomly and checks that the software never fails.
• we need to test the software as though a normal user is using it and check whether the necessary error messages,help..etc are provided.
• We need to test the software by simulating the actual Environment.For example,if a database applicaion has to be accessed by 100 users simultaneously,it is not enough if you test the software for 2 or 10 users and declare that the software is working fine.
  

To summarize,testing is difficult,To carry out the testing in alimited time with limited effort and resources,and try to bring out all the possible hidden defects is a challenging and intelligent tasks.

Types of Testing:

1.Block Box Testing
2.White Box Testing