Tuesday, 8 August 2017

How to test Machine Learning Systems?

I was working on analytical project in Healthcare some time back and heard a term used in development for data analysis – Descriptive analysis and Predictive Analysis. I asked one of my directors and he explained it in simple words which helped me to understand the basics. 

Descriptive analysis is summary of a given data set and Predictive analysis is predicting future events with the help of current data set. I was thrilled and wanted to work on predictive analysis.
Today, we implement predictive analysis using many things like data mining, statistics, machine learning etc. So this post takes us through what Machine Learning is, and more important how to test Machine Learning Systems.

Let’s see, how we do the testing in general. In case of functional testing and black box testing, we first understand the requirements. Then we create test scenarios at high level including positive and negative. We write test scripts with test steps to perform some actions. We create test data to support execution of scripted test steps. Very simple. So here we know what the expected result is? Hence in our test steps we have validation steps to check if the expected result is achieved. Can we test something where we don’t have set of expected results, answer is NO.
Now what changes here, in machine learning systems; is, we provide the lessons to the systems to perform certain actions based on the lessons. So machines will learn the lessons and perform actions based on the learning from previous run. Each time we execute; different results are expected. Now we need to test, is it performing as expected? Is it learning from its previous run?
Here, we need to understand the algorithm and mathematical formula used by the system to generate results. The testing will be based on the algorithm and not on the straight forward expected results. Each time we may get different results or incorrect results.
Following are the few pointers, testers need to remember –
1.       Understand the architecture and algorithm using mathematical coefficient to find out the working of the system, expectation is to provide the inputs after testing to modify the algorithm if required
2.       Align with business on the acceptance criteria. The results will have acceptable number of failures which business should be aligned
3.       Always create new test data and don’t use the same test data. If test data repeated, the execution will not produce the likely results. If possible automate test data generation to keep the good number of test data ready.
4.       Do not use results as it is in any of the communication, provide in depth analysis of the results. No one is expecting all pass results from the communication but more analysis on the behaviour of the system.

In upcoming posts, we will explore more on the machine learning open source tools. Stay tuned. Happy Machine Learning!!

Friday, 28 July 2017

How to test EDI transactions?

One of my friends asked me about the EDI transactions, so I thought of writing on testing EDI transactions-

What is EDI?

Electronic data interchange (EDI) is the computer-to-computer exchange of business documents between business entities. Instead of sending documents by fax or email, EDI is simple and digital way of communication.
EDI documents use specific record formats that are based on widely accepted standards. However, each company will use EDI in different ways to fit the business needs.
EDI is used in many different industries to improve efficiency. Computer based information exchange is less expensive than faxing or mailing paper documents. The processing cost and processing time is less in case of EDI.
Trading partners are the business partners who are involved in the transactions like generating purchase order, sending purchase order etc. In healthcare industry, healthcare providers are trading partners.
For successful implementation of EDI, trading partners should agree to the company about the specific formats under EDI and should implement the same. This is called EDI complaint trading partners.
EDI standards are as follows-
                                   1.    ANSI ASC X12
                                   2.    EANCOM
                                   3.    UN/EDIFACT
                                   4.    HIPAA
                                   5.    ODETTE
                                   6.    RosettaNet
                                   7.    SWIFT
                                   8.    Tradacoms
                                   9.    VDA
                                  10.  VICS

Here, we will go through ANSI ASC X12 EDI Documents-

X12 is a messaging standard developed by the American National Standards Institute (ANSI).
An X12 EDI document contains the following mandatory segments:
ISA Segment – Starting Segment
GS Segment – Starting Segment
ST Segment – Starting Segment
SE Segment – Corresponding ending segment of ST
GE Segment – Corresponding ending segment of GS
IEA Segment – Corresponding ending segment of ISA

Enveloping structure –


Within file any of the loops Starting and Ending segment can be repeated. Within ST-SE the transaction data elements are available. E.g. Invoice number / claim amount etc.
Transaction types like claim, remittance, Enquiry etc are denoted by transaction numbers and available in the implementation guide.

How to test –

The key of testing EDI transaction is to prepare test data. As shown above, we need to create test files with correct structure. These are nothing but a text files and easy to handle. Once the required files are created then process them through the system build by developers and validate the results. In general, request is sent as a file the response is created as a file. In testing, we check the response file structure if it is correct, if any data is missing etc.
Usually this testing is test data intensive and hence creating test files for all different scenarios is critical.

For more information on latest news on X12 visit the following link-

Wednesday, 21 June 2017

Next Generation Automation - Bots

I was sitting in my room and watching TV, I saw advertisement where the actor said "Alexa, reorder flour". I jumped out of chair and just said - "New Generation Automation has arrived, and this time in our day to day lives".

This not only changed my mind but changed my search and blog for this month. I checked Amazon- Alexa, Apple- Siri ,Windows- Cortana and Google - ; these all are personal assistants to provide the requested information.

Aha.... I have decided to write something on "ChatBots with IBM Watson"

Lets create simple ChatBot with Watson. We need to create IBM - Bluemix account. This is free 30 days trial. The account creation is very simple.
Enter your email, Name, password and location and submit. From email box activate the account and login.

Once logged in, it will ask to create an organisation and space. Click on "I am Ready"

1. Click on "Services"
2. Click on  "Watson"
3. Click on "Conversation"
4. Click on "Create"
5. Click on "Launch Tool"
6. Here we can create our own work space - click on Create button
7. Create Intents now - Click on "Create" (Intent is nothing but the #<<servicename>> e.g. #sales)
8. Add Intent name and examples
9. Create Dialog and we are ready

Now click on "Ask Watson". That's it we are ready with our first ChatBot, its very simple, isn't it?

You can visit for more details-


Let start new journey of Bots!!

Sunday, 30 April 2017

Open Source Automation tool - SAHI

Hi Friends,
Thank you for visiting my blog. Today, lets see another open source tool -SAHI.

 SAHI is a web automation tool which is easy to use and helpful for cross browser as well as multi browser testing. SAHI works with any installed browser which supports JAVA scripts. It helps automating web applications which have AJAX and Dynamic web elements.

SAHI has a dashboard i.e. friendly user interface to Select available browsers and links to various operations. SAHI controller is useful for record and play back. It uses SAHI, JAVA or Ruby for the same.

The idea of SAHI is to inject JAVA script into webpages and create event handlers. SAHI uses own wrappers to identify elements in DOM.

SAHI helps creating functions from the recorded script with just a click on the button. Creating library files for our automating test becomes easier.SAHI scripts can be executed on one browser, multiple browser or even on distributed machines. The most interesting part of execution is parallel run of the automated scripts. This feature makes it far more advance to test multiple scripts at the same time.

This tool automatically adds wait for AJAX and dynamic controls so that the script will work smoothly. SAHI Pro also works with FLEX controls on the web pages. Screenshot capture is an inbuilt feature of tool hence no need to write a single line of code.  Also, it provides all types of reports in form of Excel and HTML. Email feature can be used to send out reports after each execution.

SAHI provides ANT script which we can add to deploy and can integrate with JENKINS like CI tools. This integration is simple.

SAHI OS is free for lifetime, however SAHI Pro is free for one month only. SAHI Pro costs $695.

The installation and setup is simple. Please find a free demo and check the tool on the following links -


Sunday, 15 January 2017

Selenium for Desktop Application - Winium

Wish you all Happy New Year 2017. This year, I am planning to bring more tools, techniques and frameworks for all my readers.

I have few requests around the Windows based application automation which will be as simple as Web based automation using Selenium. Hence I thought of writing this post on "Winium" which is  "Selenium for Desktop Application - Winium"

Winium is simply Selenium Remote WebDriver  implementation for testing Windows based application. Winium can be useful for Windows based application based on WinForms and WPF platforms. We need other drivers as well if the Application under test is Silverlight application. However the control positions can be the best option for some of the tricky applications.

So when we are working on the complex applications pool under test and do not want license tools to complete the test automation requirement for Windows, Web and Mobile then we can consider Selenium for all in different forms. Hence for Windows - Winium, Web - Selenium WebDriver and for Mobile Appium.

As Selenium supports Java, C#, PHP, Ruby, Python like languages, Winium also supports. As we have seen that Winium desktop driver implements Selenium Remote WebDriver and listens for JsonWireProtocol and use Winium Cruciatus, Winium Cruciatus is a C# framework for WinForms and WPF Platforms.

Now, we have option to use core C# framework using Winium.Cruciatus or Selenium WebDriver remote using Winium.Desktop. Here the difference is the prior helps to write a code in C# using System.Windows.Automation namespace and the later helps to write the automation script in Selenium supported languages.

As WebDriver implementation is simple, lets talk about core C# implementation using Winium. Cruciatus wrapper on the System.Windows.Automation namespace.  First we need to add the reference to the project and then we need to create the mapping. Once the mapping  is created use the mapping in tests. Its simple.

Please refer example here -

Happy Automating Desktop Applications!

Sunday, 20 November 2016

Selenium Automation Framework

Selenium automation is one of the biggest favourite for automation testers. One of the key factor to deliver the automation work is to build the required framework which will reduce the time to maintain the automation scripts.

Lets find out what is Selenium automation framework in this post.

Selenium Automation Framework is setting standards to make the development, maintenance and operations simple. In other words any one who wants to run the scripts and analyse the results should not see any complexities in operations. All the complexities will be hidden in a logic and the libraries.

The first step is to find out about the application feasibility. This is very essential, remember Selenium works only with web applications and any windows based alerts will not be handled using webdriver. Autoit kind of tools are required to handle such alerts. So for any project to be successful the feasibility  study is must.

Next important step is to study the application flow. Straight forward steps and exceptional steps will be noted down and segregated. The requirement of test data for the steps are identified. Such data is more and follows the same steps each time then we will start thinking about data driven automation. Data driven steps are identified if certain type of data decides the flow and with all different sets of data the same steps are executed which results in different sets of outcome requires data driven framework.

We also need to decide how the reporting should be done. Most simple reports are biggest advantage of framework however they are most complex to build in many cases. The tools for reporting like TestNG and Junit are widely used. However if you have enough time and demand for some specific reports then HTML reports are very useful.

Now we know the framework for the project and reporting needs. We can start building library functions. These functions are as simple as opening client site, clicking on button and entering data in text field. The basic control should be targeted first.

Another important step is to create the driver and decide how do you want to call the scripts. All the library functions and client scripts should take values from property files only. This way we can make sure that the same function can be used for different operations.

Hope the above information gives basic idea of the selenium automatio framework. I will be waiting to hear from all of you.

Happy framework designing.

Saturday, 12 November 2016

Security Testing

This post is dedicated to the specialised branch of software testing - Software Security Testing.

Before we get into Software Security Testing which is always part of Non Functional Testing unit we will see what are the types of Software Threats -

1. Brute Force Attack

Brute Force Attack is a vital information guessing technique. In this process automated system generates many combinations of user password or pin (personal identification number) to get into the business machines with the intention of stealing the data. These attacks are crafted very carefully so that automated system should get the access to the business systems quickly.

2. Denial of Service

This type of attack is on the critical business systems like bank, payment gateways, booking sites or critical news boards. The intention of this attack is to stop users to access the application. The attack is usually  done by the automated system by creating virtual users in exponential process with the intention of blocking available bandwidth of server to deny access to any users who need information.

3. SQL Injections

SQL Injections are the SQL queries to read, modify or delete business critical information stored in the database  using presentation layer. These attacks are easy and can be planned quickly. However these attacks are most of the time risky attacks and easy to track down.

4. Trojan Attacks

Trojan attacks are most common and easy to find. In these attacks users are presented with some of the interesting tasks like disk back up, disk fragmentation or memory management and at the background  automated scripts are executed to steal information. These attacks are successful if the user follows instructions given on the websites without confirming with the security teams.

5. Fishing

These attacks are usually crafted by sending out emails to mass users. These emails are lottery, charity or some business proposals. Users fall for this type of invitations because the offer is very lucrative and directly comes to the email inbox. The email comes with some kind of URL to hit or buttons or images to click the confirmation so that users get trapped.

Now we understand the possible attacks and threats to the business so the first task is to create test strategy to find out, in our client's business such vulnerabilities are present and if present raise an issue to fix them before the business application starts handling the user information or business critical information.

Except the fishing attack all other types of attacks are tested using penetration tools. For fishing attack we usually inform the staff and add email scanners to scan the emails coming into the business network and going out of the business network.

Open source tools used for testing above attacks are Aircrack-ng, Automatic backholing, SQLMap, pfSense and Gophish respectively.

Enjoy Security Testing!