Some early cloud adopters have had a happy and successful cloud project experience marred down the line by testing...
and application lifecycle management issues. A successful project is one that meets the business goals as well as security and compliance requirements.
To be sure that's all accomplished, cloud planners, architects and developers should start with ensuring that test data and processes validate the user experience and visualize the technology groups within their cloud deployment. They should vertically integrate user experience with security and compliance in each group, and horizontally integrate within and between groups using proven workflow techniques that can be effectively tested.
Effective test data generation for the cloud has to follow the entire path of real user data and provide for response time measurement and reporting, or there's no way to determine whether the top-level user experience is even being tested. This is a particular challenge for mobile applications, which are increasingly the prime target for cloud deployment.
There are two ways to couple test data effectively:
- Introduce data at multiple points just as mobile users would
- Measure mobile data variations independently and combine the results with application performance data to assess response time and QoE
It is very difficult to drive a realistic user-to-application-to-user workflow in the cloud without making the test process into a cast-of-thousands effort. It's smarter to take user front-end access statistics and combine them with the rest of the testing and ALM techniques. That means getting user access statistics from the widest possible user distribution from other applications. Combining this with testing and ALM QoE statistics is easiest if you design your cloud application with a test GUI hook into the front-end process, something that parallels the real user connection and duplicates its internal processing delays to the greatest extent possible.
Testing and ALM middleware tools
Another aspect of cloud applications that complicates testing is the variability of middleware and platform tools across the various elements of the application. Most cloud applications have a front-end GUI process that feeds a deeper transaction processing element. The front end is often based on a Web-oriented technology like Java and the back end might be traditional SOA using a service bus. While it's important to test through this entire flow, it may be helpful to test each segment independently to spot problems and aid in isolation and repair.
If you group your application components by middleware, you may be able to define two or three technology groups within your application. In most cases these technologies won't be distributed randomly through the application but will be exercised on a linear path from worker to database. Application testing during development, and ongoing ALM, can exercise these technology groups first to insure they're functioning according to specifications, then do end-to-end testing aimed primarily at validating overall QoE and functional coupling and integration. To do this, each of the technology groups should be vertically linked with the functional business requirements it's contributing to, and the security and compliance issues that it must address.
The technology-group approach may be particularly helpful in adapting existing ALM techniques to new cloud deployments because the back-end technology groups are usually largely unchanged in cloud applications. Those cloud changes focus on the user front-end processes, and these new processes are also more likely to introduce new security and compliance issues. Testing at this level should validate that the back-end processes are unchanged (or accommodate minimal changes actually made) and then move to validating the behavior of the front end. The fact that a technology group was selected based on harmonized middleware choices facilitates both tool selection and testing.
For private cloud and Docker or container users, a technology cluster is also a good place to start portability and scaling plans. The technology cluster is harmonious in terms of OS and platform, so it can be mapped to a single resource pool or become the basis for a Docker Swarm. Horizontal scaling and failure recovery are then facilitated and the standard framework you've built this way will be easier to test.
Horizontal integration is normally the focus of both testing and ALM techniques. For the cloud, it's often easiest to first do horizontal integration testing within a technology group by following the workflow through that group of components. Most issues with security and compliance can be picked up at this level of testing, and it's simpler to conduct the tests on a group of components than across the entire workflow. It's probably not wise to try to do QoE testing by adding process times for all the groups; the relationship between group times and total time is more complicated and end-to-end tests of performance can be done as the final step.
When doing horizontal integration testing, the primary goal should be to ensure that all the variants in component positioning and deployment are tested. This can be difficult with public cloud services because you probably won't have the ability to force deployment changes in the basic cloud services. It's best to contact your public cloud provider(s) to get assistance on how to ensure you can deploy components in all the possible hosting points for testing.
Most security and compliance problems generated by technology or design flaws will creep in because of flaws in cloud integration. Often the processes of locating deployed or moved components open flaws in the security of the directory system being used, and in many cases simple testing is unlikely to uncover these issues. A thorough design review should be conducted prior to the first field tests of the application, and any changes to directory management should be made only after another such review.
The cloud doesn't need new testing tools as much as it needs cloud-centric testing planning. This should start with application planning and not when you're ready to test. Carefully blending knowledge of the cloud with knowledge of the application allows you to organize both application testing and ALM techniques to their optimal state for your business.
A guide to cloud testing and ALM
Expediting Agile testing and ALM
Four categories of application testing tools