Testing Times!!! Monkey Theorem and Automation

I almost always wonder if all testers are paranoid. What if it stops working in this way, what if something is missing here, what if it is easily manipulated, and so on with never-ending what-ifs.

In today’s time, what one commonly sees is this paranoia. People all over the world are double-checking, confirming to ensure they wear masks, carry sanitizers, and “testing” if they are COVID negative. Better to be sure and safe than sorry, right! Friends, often testing software applications would require such a mindset even if it is formal testing (structured) or informal (exploratory) testing to avoid a vulnerability later.

The fun part of testing is finding a hidden problem. And, it is important to keep yourself distant. If you are too close to it during development, it becomes even difficult for you to find the problem. I would recommend that you have a fresh set of eyes before testing, so you don’t have a blind spot. Especially while adding new features, it is not only important to validate the current feature, but also to make sure nothing in the existing set of features break.

We cannot solve our problems with the same thinking we used when we created them.” — Albert Einstein

Testing over the years has evolved from using the standard waterfall approach to automated tests to Agile testing (faster releases), RAD to DevOps/Continuous testing/CICD approach, Cloud testing, and now to Autonomous testing, Machine Learning and AI.

Have you heard of the Infinite Monkey theorem? I found this analogy very interesting.

The infinite monkey theorem states that a monkey hitting keys at random on a typewriter keyboard for an infinite amount of time will almost surely type any given text, such as the complete works of William Shakespeare.

Automated scripts could be compared to the monkey. Here, the monkey is not intelligent. If you are implementing a new feature, then validating that feature thoroughly is not enough. “Just because you’ve counted all the trees doesn’t mean you’ve seen the forest.”. However, checking the whole application to see if all is working as expected with the new feature in place would be tedious. Here is where automated scripts help you, that is in doing “Regression Testing” better termed as Version Control.

We have heard and used competing automation tools over time. QTP, Selenium, Cypress, Puppeteer, Playwright being some of them.

Some of the challenges with automation testing are the use of optimal waits, parametrization, and maintainability.

Feed context to the monkey, train the monkey, make it intelligent, you get AI.

With AI, we can find a solution to these challenges. If an id or Xpath of an element changes, the whole script fails. With the help of dynamic locators, AI understands that if the name of an element or id of an element is changed, the functionality hasn’t changed, just the attribute is changed. Then it searches for the next attribute, say tag name, and if that is also changed, then it searches for the next attribute, say the class name, and so on thus not failing your test.

The wait issues, we often experience, where for one page or button to be displayed, it takes different response times, sometimes the response from the server may take only 2 sec, another time, it might take 15 sec to load. Here, we would have to optimize wait. With AI, we would not have to worry about that, as the time and response from the server every time we ping the server is recorded and then that is relayed for further tests without having to go the server. This would speed up your UI based testing.

Basically, AI has its own self-healing mechanism.

The test coverage is increased. As applications get bigger and bigger, it becomes increasingly difficult, even for experienced programmers to recognize every interconnection and I guess an intelligent tool can help with that.

 

Continue reading...