Dynamics AX Performance Testing by using Coded UI tests

Recently we have completed Dynamics AX performance testing by using functional test automation scripts, which were developer by using Visual Studio - Coded UI. Dynamics AX thick client performance testing was time consuming using X++ code approach - Benchmark Toolkit. I'm into Test Automation for many years and had few challenges to implement functional test scripts as performance Testing. Used Microsoft Terminal servers to simulate multiple users RDP sessions. Prepared the list of factors, which are considered as helpful in this methodology.

Best practices for coding

• Don't open the AX thick client for each iteration. For example, Sales Order creation should be executed for 15 times per user. Open AX thick client only once per user and then create 15 Sales order without re-starting AX client
• Open AX client from windows path. Don't use shortcuts to open. Also keep the same path should be available in all terminal servers.
• Methods to log Start Time and End Time for particular transaction
• Don't use too much descriptive OR heavy programs to find the controls. Keep in mind that Testing tool also will consume CPU and memory
• Don't keep control identifying calls within the time measuring statements. There is a major difference on purpose of functional script and performance script.
• You can use UI Map (Object Repository) OR Page object model
• Try to use short-cut keys. It would simplify your script as well as increase the execution speed
• Introduce the loop within the test method. Don't call test method multiple times
• Use data sources to provide different test data for each user. Try to read once and then use, whenever it required
• Implement time logging only at essential code. Also consider that Automation tool would take few seconds to identify particular screen/UI object
• Run with few users and validate once the script/ scenario is complete
• Avoid left click navigation by entering module directily in address bar
• Use Sleep between transactions, but do not include in transaction timings
• Refine the steps, wherever possible
• Follow naming conventions for transaction name, which should be simple and ease of use

Best practices for Performanc Testing related

• Transaction timings can be logged as XML, TXT, DB records etc. I would prefer to keep in Database. SO that you can save the data multiple times and also use queries to collect different kind of metrics like Average Response time, 90th Percentile response time, Minium Response time, Maximum Response time etc.
• Create couple of methods to log start and end timings for each transaction
• Capture the script errors as well as AX exceptions
• Update the information for
• Implement test method to be executed for the given time using config file. For example, the scripts have to be executed 60/90/120 minutes
• Implement user frequency. For exmple, if Sales Order should be created 15 times per user, then script should be stopped after creating 15 Sales order for each user
• Capture screenshots if test is failed for any user. Keep the images in shared path. Also create less size image.
• Modify the script, which is creating more transactions than expected
• Validate each transaction time as whether only applicaiton time OR included Coded UI time as well
• Keep data files in the shared path for ease of use and maintenance
• Use Visual Studio Load test to collect performance counters from various servers like AOS servers, AX Batch servers, AX Integration servers, Terminal servers etc

Batch files to execute CodedUI Tests
Always create DLLs in Release mode instead of Debug mode. It will improve system's resource utilization.

 
echo on
set logfile=C:\AX_PerformanceTest\ExecSmallSO.log
echo "%date% - %time%- MediumSO started"
cd\

cd "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow"

vstest.console.exe "C:\AX_PerformanceTest\DLL\AX_PerformanceTest.dll" /Tests:CreateSmallSaleOrder

pause

Sample Coded UI methods
Scenarios should be carefully implemented as it is for performance testing.

 
        public void clickSellTab()
        {
            axAppWindow.SetFocus();
            myutils.ClickButton(axAppWindow, "SellTab");
        }
  
  public void ClickBookMenu(ref Logger.Logger LogObj)
        {
            try
            {
                axAppWindow.SetFocus();
                WinMenuItem item = new WinMenuItem(axAppWindow);
                item.SearchProperties[WinMenuItem.PropertyNames.Name] = "Book";
                UITestControlCollection ab = item.FindMatchingControls();
                LogObj.TransactionStart("BookingOrder");

                Mouse.Click(item);
                Thread.Sleep(1000);
                this.clickSellTab();
                // axAppWindow.WaitForControlEnabled(50000);
                axAppWindow.SetFocus();
                item.WaitForControlReady(50000);
                LogObj.TransactionEnd("BookingOrder");
            }
            catch (Exception ex)
            {
                LogObj.LogException(LogObj.ScenarioName, ex.ToString());
            }            
        }

Few links related to Coded UI Tests

• Coded UI Testing Best Practices for Microsoft Dynamics AX
• Hybrid Coded UI Testing
• Data driven Coded UI test with Dynamics AX

Sharepoint Test automation using CSOM

Microsoft has provided Client Side Object Model (CSOM) to interact with Sharepoint objects easily. This CSOM objects can be interacted through Powershell and C#. Many developers are also using this to do CRUD[Create, Read, Update and Delete] operations. Testers can use CSOM to validate site columns, various content types and content data. Also it can be used to create test data for various content types.

I have used this library - SharePoint Test Automation using Client Side Object Model (CSOM) for my sharepoint projects. Used to validate site columns, available sites content types and content fields.

Follow the steps to begin your automation. Also it is API-based automation, no need to worry about UI changes...

Step 1: Download and Install Client Components from SharePoint Test Automation using Client Side Object Model (CSOM)

Step 2: Once installed, Add the following references in the package.
- Microsoft.SharePoint.Client
- Microsoft.SharePoint.Client.Runtime

Step 3: Develop code for Test Initialize like below.

 
using System;
using System.Configuration;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Test.Automation.SharePointCOM.Verification;
using Test.Automation.SharePointCOM.Helpers;
using System.Net;
using Microsoft.GD.Test.Logging;
using System.IO;
using Microsoft.SharePoint.Client;
using AutomationFramework.TestScripts;

namespace AutomationFramework.TestScripts
{
    [TestClass]
    public class SpAppVerify
    {
        
        SPVerification VerifyCO;
        LogFile TestResults = new LogFile();
        private TestContext testContextInstance;
        public static string sURL = "https://teams.SpApptest.com/sites/cthub";

        public static string sUsrId = "svc-spfarm"; 
        public static string sUsrPwd = "SpAppdev#123@"; 
        public static string sUsrDomain = "devSpAppnai"; 
        private String sTestLogFile = ConfigurationManager.AppSettings["ResultsLog"];

        public TestContext TestContext
        {
            get
            {
                return testContextInstance;
            }
            set
            {
                testContextInstance = value;                
            }
        }


        [TestInitialize]
        public void Setup()
        {
            // The below two lines help to access HTTPS site
            System.Net.WebClient client = new System.Net.WebClient();
            ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
            VerifyCO = new SPVerification();
            testContextInstance.WriteLine("==============Starting the Set Up=====================");
            Assert.IsTrue(VerifyCO.SetSiteUnderTest(sURL, sUsrId, sUsrPwd, sUsrDomain, testContextInstance));
            testContextInstance.WriteLine("==============Ending the Setup=====================");
            
        }

Step 4: Site Column validation

 
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"C:\Automation\SpApp_Tests\CoreLibrary\TestData\sitecolumns.csv", "sitecolumns#csv", DataAccessMethod.Sequential), TestMethod]
        //[TestMethod]
        public void VerifyCOSiteColumns()
        {
            TestStartMessages(testContextInstance);
             VerifyCO = new SPVerification();
            String ColumnName = testContextInstance.DataRow["SitecolumnName"].ToString();
             VerifyCO.SetSiteUnderTest(sURL, sUsrId, sUsrPwd, sUsrDomain, testContextInstance);
             TestResults.LogMessage(testContextInstance.TestName + " SitecolumnName : " + ColumnName);
            TestResults.LogAssert(VerifyCO.VerifySiteColumns(ColumnName, testContextInstance, "SitecolumnName", "SpApp Site Columns"));             
        }

Step 5: Content Type fields validation

 
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"C:\Automation\SpApp_Tests\CoreLibrary\TestData\ContentTypes\SpApp Web Page ", "SpApp Web Page#csv", DataAccessMethod.Sequential), TestMethod]
        public void SpApp_Web_Page()
        {
            TestStartMessages(testContextInstance);
            VerifyCO = new SPVerification();
            VerifyCO.SetSiteUnderTest(sURL, sUsrId, sUsrPwd, sUsrDomain, testContextInstance);
            String TypeName = testContextInstance.DataRow["ContentTypeName"].ToString();
            String ColumnName = testContextInstance.DataRow["ColumnName"].ToString();
            String Field = testContextInstance.DataRow["Type"].ToString();
        //    Assert.IsTrue(VerifyCO.VerifyContentTypeField(TypeName, ColumnName, Field, testContextInstance));
            TestResults.LogMessage(testContextInstance.TestName + " : " + TypeName + " : " + ColumnName + " : " + Field);
            TestResults.LogAssert(VerifyCO.VerifyContentTypeField(TypeName, ColumnName, Field, testContextInstance));  
        }

Step 6: Bonus code - To get all fields of content types in the given site.

 
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"C:\Automation\SpApp_Tests\CoreLibrary\TestData\AllContentTypes.csv", "AllContentTypes#csv", DataAccessMethod.Sequential), TestMethod]
        public void Get_All_ContentTypes()
        {           
            bool result = false;
            String sContentInfo;
            FieldCollection Fields;

            TestStartMessages(testContextInstance);
            VerifyCO = new SPVerification();
            VerifyCO.SetSiteUnderTest(sURL, sUsrId, sUsrPwd, sUsrDomain, testContextInstance);
            
           // SharePointHelper spSharePointHelper = new SharePointHelper(sURL, sUsrId, sUsrPwd, sUsrDomain);
            String TypeName = testContextInstance.DataRow["ContentTypeName"].ToString();
            Fields = VerifyCO.GetAllContentTypes(TypeName);          

            if (Fields != null)
            {
                TestContext.WriteLine("Found the Content Type : " + TypeName);
                TestResults.LogMessage("Found the Content Type : " + TypeName);
                // FieldCollection Fields = spSharePointHelper.GetFieldsOfContentType(TypeName);

                foreach (Field fieldItem in Fields)
                {
                    sContentInfo = "Title: " + fieldItem.Title;
                    sContentInfo += " DisplayName: " + fieldItem.TypeDisplayName;
                    sContentInfo += " Description: " + fieldItem.Description;
                    sContentInfo += " Default Value: " + fieldItem.DefaultValue;
                    sContentInfo += " Required: " + fieldItem.Required.ToString();
                    //sContentInfo += "DisplayName: " + fieldItem.TypeDisplayName;
                    TestResults.LogMessage(sContentInfo);

                }

                
            }

            else
            {
                TestResults.LogMessage("Not Found Content Type : " + TypeName);
                TestResults.LogMessage("  ");
            }
            
        }

Links For Sample code

Using the C# CSOM to interact with SharePoint Online

Use C# CSOM ClientContext to download file from SharePoint document library or OneDrive for Business

CodedUI Useful links

In recent yearts, CodedUI has been improved lot. Using latest version of VS, we can automate windows apps and windows phone apps. Below links are giving more info and how to use Coded UI for different applications.

CodedUI Resources
Verifying Code by Using UI Automation
Hand Coding a CodedUI test
How To: Automating Custom controls through CodedUI

Copy all comments from MS Word document

I tried today only and it worked. Go to your first comment and select all the text. Press SHIFT key and DOWN arrow key simultaneously up-to reaching the last comment. Then Copy (CTRL+C) and Paste (CTRL+V) in any text editor. All the comments would be available. Saving Time..

Logging utility in C#

While doing Sharepoint CSOM Automation, I have written a simple utility to create test logs. Usually CodedUI results are not stored and this utility used to capture for each run. Also trying to extend the Assert function for capturing exceptions.

 
namespace AutomationFramework.TestScripts
{
    class LogFile
    {
        private String sLogFile = ConfigurationManager.AppSettings["LogResultsFile"];

        public void LogMessage(String sMsg = " ")
        {

        // Create a writer and open the file:
        StreamWriter log;

        if (!File.Exists(sLogFile))
        {
          log = new StreamWriter(sLogFile);
        }
        else
        {
          log = File.AppendText(sLogFile);
        }

        // Write to the file:
        log.WriteLine(DateTime.Now + " : "+ sMsg);
             
        // Close the stream:
        log.Close();
        }

         public void LogAssert(bool bCondition, String sMsg=null)
         {
             bool bResult = false;
             try
             {
                 
                 Assert.IsTrue(bCondition,sMsg);
                 bResult = true;
             }
            catch (Exception ex)
            {
                LogMessage("Exception: " + ex.Message);
                Assert.Fail("Test FAILED");
            }
            finally
            {
                if (bResult)
                {
                    LogMessage("PASS ");
                    
                }
        }
       }
}