Quick Time to Market with high quality for a global leader in VPN services software platform

Quick Facts

200 tests execution / in 30 min / in parallel/ 5 platforms: iOS, Android, macOS, Windows, Linux

95% of test runs have no failures caused by unreliable tests

All test runs use reproducible environments

Test Automation integrated in CI

Objective

To increase product scalability with huge global customers base, capability, and accelerate time to market.

Challenge

Intetics’s client, a global leading technology company, provides most trusted and stable VPN access for customers all over the world. They decided to increase their test visibility and scalability to update their product faster than competitors. The main challenges were related to quickly responding to new firewalls created by censorship countries, like China, and also be faster than competitors.

To address the needs of the specific audience, the company decided to fix all bugs and respond to threats very fast.

The client needed a profound platform assessment which would allow to perform automatic quality control on a permanent base and to exclude manual work. The client chose Intetics for its extensive expertise in QA and software products assessment and requested to carry out an unbiased analysis.

Solution

To launch the assessment process, the Intetics team studied the client’s platform infrastructure, discussed with the product owner the assessment approach, set up the reporting deadlines and agreed on frequency of test automation execution.

After the initial assessment, Intetics team took part in building a test automation system to run auto-tests across all platforms – Android, iOS, Mac, Windows, Linux. It allowed to ship and react quickly on all incoming challenges.

Every assessment stage ended up with a report describing the product weaknesses and bottlenecks in details. The whole process took 4 months. The final report demonstrates the overall system state as medium and requiring tech enhancements.

#1 Make test results more visible

Automation automatically triggered a github commit, and results became clear and informative

#2 Work together, not separately

Developers and manual QAs were included in process of creating, updating and running automation tests

#3 Run significantly more tests

Number of test cases increased by improving test speed and decreasing time-consuming steps

#4 Test in more environments

Test cases which emulate edge cases, poor network issues and censorship environment were added, so such problems are faced and fixed much faster now

#5 Parallel execution

Tests cases are designed for parallel run to split them between different agents and run faster

#6 Hardware Architecture 

Different types of hardware are used depending on platform like emulators, real devices, Mac and Windows VMs, running in clouds.

Technologies

Benefits and Results

Eliminated distance between development and QA teams

Test frameworks were created across all platforms

Test coverage, trustability & durability were improved

Product scalability was enhanced