Dynamics AX Performance Testing by using Coded UI tests

Recently we have completed Dynamics AX performance testing by using functional test automation scripts, which were developer by using Visual Studio - Coded UI. Dynamics AX thick client performance testing was time consuming using X++ code approach - Benchmark Toolkit. I'm into Test Automation for many years and had few challenges to implement functional test scripts as performance Testing. Used Microsoft Terminal servers to simulate multiple users RDP sessions. Prepared the list of factors, which are considered as helpful in this methodology.

Best practices for coding

• Don't open the AX thick client for each iteration. For example, Sales Order creation should be executed for 15 times per user. Open AX thick client only once per user and then create 15 Sales order without re-starting AX client
• Open AX client from windows path. Don't use shortcuts to open. Also keep the same path should be available in all terminal servers.
• Methods to log Start Time and End Time for particular transaction
• Don't use too much descriptive OR heavy programs to find the controls. Keep in mind that Testing tool also will consume CPU and memory
• Don't keep control identifying calls within the time measuring statements. There is a major difference on purpose of functional script and performance script.
• You can use UI Map (Object Repository) OR Page object model
• Try to use short-cut keys. It would simplify your script as well as increase the execution speed
• Introduce the loop within the test method. Don't call test method multiple times
• Use data sources to provide different test data for each user. Try to read once and then use, whenever it required
• Implement time logging only at essential code. Also consider that Automation tool would take few seconds to identify particular screen/UI object
• Run with few users and validate once the script/ scenario is complete
• Avoid left click navigation by entering module directily in address bar
• Use Sleep between transactions, but do not include in transaction timings
• Refine the steps, wherever possible
• Follow naming conventions for transaction name, which should be simple and ease of use

Best practices for Performanc Testing related

• Transaction timings can be logged as XML, TXT, DB records etc. I would prefer to keep in Database. SO that you can save the data multiple times and also use queries to collect different kind of metrics like Average Response time, 90th Percentile response time, Minium Response time, Maximum Response time etc.
• Create couple of methods to log start and end timings for each transaction
• Capture the script errors as well as AX exceptions
• Update the information for
• Implement test method to be executed for the given time using config file. For example, the scripts have to be executed 60/90/120 minutes
• Implement user frequency. For exmple, if Sales Order should be created 15 times per user, then script should be stopped after creating 15 Sales order for each user
• Capture screenshots if test is failed for any user. Keep the images in shared path. Also create less size image.
• Modify the script, which is creating more transactions than expected
• Validate each transaction time as whether only applicaiton time OR included Coded UI time as well
• Keep data files in the shared path for ease of use and maintenance
• Use Visual Studio Load test to collect performance counters from various servers like AOS servers, AX Batch servers, AX Integration servers, Terminal servers etc

Batch files to execute CodedUI Tests
Always create DLLs in Release mode instead of Debug mode. It will improve system's resource utilization.

 
echo on
set logfile=C:\AX_PerformanceTest\ExecSmallSO.log
echo "%date% - %time%- MediumSO started"
cd\

cd "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow"

vstest.console.exe "C:\AX_PerformanceTest\DLL\AX_PerformanceTest.dll" /Tests:CreateSmallSaleOrder

pause

Sample Coded UI methods
Scenarios should be carefully implemented as it is for performance testing.

 
        public void clickSellTab()
        {
            axAppWindow.SetFocus();
            myutils.ClickButton(axAppWindow, "SellTab");
        }
  
  public void ClickBookMenu(ref Logger.Logger LogObj)
        {
            try
            {
                axAppWindow.SetFocus();
                WinMenuItem item = new WinMenuItem(axAppWindow);
                item.SearchProperties[WinMenuItem.PropertyNames.Name] = "Book";
                UITestControlCollection ab = item.FindMatchingControls();
                LogObj.TransactionStart("BookingOrder");

                Mouse.Click(item);
                Thread.Sleep(1000);
                this.clickSellTab();
                // axAppWindow.WaitForControlEnabled(50000);
                axAppWindow.SetFocus();
                item.WaitForControlReady(50000);
                LogObj.TransactionEnd("BookingOrder");
            }
            catch (Exception ex)
            {
                LogObj.LogException(LogObj.ScenarioName, ex.ToString());
            }            
        }

Few links related to Coded UI Tests

• Coded UI Testing Best Practices for Microsoft Dynamics AX
• Hybrid Coded UI Testing
• Data driven Coded UI test with Dynamics AX