Convert failed testcases as a testplan

We have few silktest automation projects. One project is used to execute by suite. Another one is ran by batch (*.bat) file, which has few sub-plans. I am not able to run the failed cases from all results at one shot. I was looking an utility, similar to 'Mark failures in Plan' in the results menu. I thought to implement a 4test script to convert failed testcases as a testplan and I did it.

4Test code - Create testplan for failed cases only

[ ] LIST OF STRING glsFailedScripts = {}
[ ] 
[+] testcase ConvertTestPlan_SingleFile () appstate none
[ ] 
[ ] STRING sResFile = "D:\Auto_Scripts\Results\MyStuff_MasterPlan.res "
[ ] STRING sRexFile = "D:\Auto_Scripts\Results\MyStuff_MasterPlan1.rex "
[ ] STRING sMyPlanFile =  "D:\Auto_Scripts\Results\zSingleFailed.pln "
[ ] 
[ ] glsFailedScripts = {}
[ ] CreateFileFromTemplate(sPlanTemplate,sMyPlanFile)
[ ] ResExport (sResFile,sRexFile)
[ ] ProcessSingleREXFile (sRexFile,sMyPlanFile)
[ ] 
[ ] Agent.DisplayMessage ("test","Completed.")
[ ] 
[-] // functions
[+] Boolean GetFailedTestInfo (STRING sRexLine, out LIST OF STRING lsPlanInfo)
[ ] 
[ ] // Title
[ ] //TestPlan Script TestCase TestData ErrorCount ErrorText DateTime Elapsed
[ ] 
[ ] // Samples
[ ] // "D:\Auto_Scripts\PlanFiles\MyStuff_MasterPlan.pln","D:\Auto_Scripts\Scripts\FamilyFunctions.t","FamilyFunctions_DimensionFunctions_5",
[ ] // "Verify user sees a proper error message by clicking OK button.",0,"","2008-08-18 21.52.06","0:00:00"
[ ] 
[ ] STRING sScript
[ ] STRING sTestCase
[ ] STRING sTestData,sFormattedData
[ ] STRING sTestTime
[ ] STRING sErrCount, sErrCount1
[ ] INTEGER iPos
[ ] 
[ ] // Initialization
[ ] STRING sIndentation = " "
[ ] STRING sIndentation1 = "[+]  "  // Level1
[ ] STRING sIndentation2 = " [ ] " // Level2
[ ] 
[ ] STRING sSeperator = ""","""
[ ] STRING sErrSeperator = ","""   //",""***"
[ ] STRING sDataSeperator = """,0,"""
[ ] STRING sErrSeperator1 = """"
[ ] Boolean bResult = FALSE
[ ] lsPlanInfo = {}
[ ] 
[ ] sErrCount1  = GetField (sRexLine,sSeperator,4)
[ ] sErrCount  = GetField (sRexLine,sDataSeperator, 2)
[ ] //if ((Trim(sErrCount) != "") && (Val(sErrCount) > 0))
[+] if ((Trim(sErrCount) == "")  && (Trim(sErrCount1) != "") )
[ ] Print ("Considering line: {sRexLine}")
[ ] bResult = TRUE
[ ] sScript   = GetField (sRexLine,sSeperator,2)
[ ] sTestCase   = GetField (sRexLine,sSeperator,3)
[ ] sTestTime = GetField (sRexLine,sSeperator,6)
[ ] Print ("Elapsed: {sTestTime}")
[+] if (sTestTime == "")
[+] sFormattedData   = ""
[ ] iPos = StrPos (sErrSeperator, sTestCase,TRUE)
[+] if (iPos > 0)
[ ] Print ("sTestCase:{sTestCase}: iPos:{iPos}")
[ ] sTestCase = SubStr (sTestCase, 1,iPos - 4)
[ ] Print ("Before formatting, sTestCase:{sTestCase}")
[ ] //sTestCase = StrTran (sTestCase,"\","")
[ ] Print ("After formatting, sTestCase:{sTestCase}")
[ ] 
[+] else
[ ] sTestData   = GetField (sRexLine,sSeperator,4)
[+] if (MatchStr ("*,?,""*",sTestData))
[ ] iPos = StrPos (sErrSeperator, sTestData,TRUE)
[+] if (iPos > 0)
[ ] Print ("TestData:{sTestData}: iPos:{iPos}")
[ ] sTestData = SubStr (sTestData, 1,iPos - 4)
[ ] Print ("Before formatting, TestData:{sTestData}")
[ ] sFormattedData = StrTran (sTestData,"\","")
[ ] Print ("After formatting, TestData:{sTestData}")
[ ] 
[ ] Print ("TestCase: {sTestCase}")
[ ] Print ("TestData: {sFormattedData}")
[ ] 
[+] if ( (glsFailedScripts == {}) || (glsFailedScripts[ListCount(glsFailedScripts)] != sScript) )
[ ] ListAppend (glsFailedScripts,sScript)
[ ] ListAppend (lsPlanInfo,"{sIndentation1}#script: {sScript}")
[+] // if (Trim (sTestData) != "")
[ ] // ListAppend (lsPlanInfo,"{sIndentation2}#testdata: {sFormattedData}")
[ ] ListAppend (lsPlanInfo,"{sIndentation2}#testcase: {sTestCase}({sFormattedData})")
[ ] 
[+] else
[ ] Print ("Not Considering line: {sRexLine}")
[ ] 
[ ] return bResult
[+] public void TestPlanWrite(STRING sPlanFile, ANYTYPE aMsg)
[ ] // To write content into log file
[ ] HFILE hFile
[ ] STRING sItem
[ ] 
[ ] hFile = FileOpen (sPlanFile, FM_APPEND)
[+] switch TypeOf (aMsg)
[+] case STRING
[ ] FileWriteLine(hFile,"{aMsg}")
[+] case  LIST OF STRING
[+] for each sItem in aMsg
[ ] FileWriteLine(hFile,"{sItem}")
[+] default
[ ] Print ("Non supported element for TestPlanWrite: {TypeOf (aMsg)}")
[ ] FileWriteLine(hFile,"{aMsg}")
[ ] 
[ ] FileClose (hFile)
[ ] 
[ ] 
[+] VOID ProcessSingleREXFile (String sRexFile, STRING sPlanFile)
[ ] // To create a testplan
[ ] // Implemented for Auto Scripts 
[ ] // It will read the text file 
[ ] 
[+] // Parsing REX file and then writing into plan file
[ ] HFILE hInFile
[ ] STRING sLine
[ ] LIST OF STRING lsPlan
[ ] 
[ ] hInFile = FileOpen (sRexFile,FM_READ)
[ ] 
[+] while (FileReadLine (hInFile, sLine))
[+] if (GetFailedTestInfo (sLine,lsPlan))
[ ] TestPlanWrite (sPlanFile,lsPlan)
[ ] 
[ ] FileClose (hInFile)

Test Results Reporting

Results Reporting is very much important as automating functionalities. It will be better, if results are logged with screen shots for failed cases. Most of us, not concentrating much on Results Reporting. It will improve your automation level higher.

Automation Engineers are developing lot of test scripts. How will you check all scripts results? We can't check each testcase by testcase. Summary of test results is so important. Also the compiled results for all results is good to have.

Long back, I studied one article related with Automatic Results processing. That document discussed about extracting some keywords from particular log files and store the results in different format. I have implemented using Excel sheet and it will give result for each testcase(function) as well as compiled result for each scripts. If summary and results are stored into HTML files, we can easily get to know the results.

I agree, however one person need to look the results. Some of the tools are giving summary details after execution. I want to go one step ahead. I need to see all automation results of particular project from one file (HTML or Flat file or XL sheet). It is like maintaining history.

But Results processing is very common for test automation as well as general applications. You can follow like any standard log files format. You can maintain 2-3 log files for single script. One for all the activities of your suite and another one for Just giving Pass or Fail result for each testcase. It will be better, if all the log files are HTML files. At last prepare a single file to show the compiled results for the whole execution. You can publish the results easily.