How to Do DevOps Testing with Morpheus

The widespread adoption of agile development methods has transformed the software design timeline into a “time circle.” Everything happens simultaneously: design, test, deploy, maintain. Considering the fundamental change this represents, it’s no wonder companies often struggle to implement continuous integration/continuous delivery for their vital data systems.

The area of agile development that trails all others is testing. Software Development Times cites a recent study by test tool vendor Zephyr that found 70 percent of the small companies surveyed have adopted agile processes, but only 30 percent have automated their testing operations. Another survey sponsored by Sauce Labs (pdf) reports agile adoption has reached nearly 88 percent at organizations of all sizes, yet only 26 percent of the companies have implemented test automation.

DevOps teams have to manage design, test, deploy, and maintain products while keeping in mind the needs of four distinct, diverse groups: 

• QA who are in charge of use cases, test cases, test plans 
• Developers/testers who implement the test cases 
• DevOps who integrate testing into the continuous-delivery pipeline 
• Line managers who generate and act on test reports

In a DZone post, Sergio Sacristan runs down the testing features each distinct segment of the continuous-deployment community requires. At the testing stage, the first two required characteristics are robustness and stability. Testers look for an environment that requires little training to use, that keeps development and maintenance simple, and that is easy to port between applications.

At the execution stage, the key is seamless integration of testing with deployment tools. The test environment must be flexible and adaptable to accommodate desktop apps, web apps, mobile apps, cloud services, and test scenarios. It should be easy to apply the same test with different parameters, runtimes, and application properties. Other considerations are the ability to run tests in parallel, to scale tests without sacrificing performance, and to ensure testing tools are easy to access.

The continuous-delivery component that DevOps is most likely to overlook is the one that's most critical to managers: reporting. The test environment must be capable of sending reports—whether via email or text—that communicate pertinent test results in a timely manner. After all, the primary purpose of testing is to identify and address problems. In addition to complete, up-to-date logs, the report function has to provide access to a history of executions and offer customizations to allow performance to be measured between versions.

Imagine an application deployed to production with zero defects. What many DevOps teams would consider a pipe dream is an achievable goal according to Alex Martins, who explains in a DZone post the important distinctions between continuous testing and test automation. In a nutshell, test automation is a process that is applied at some level to nearly all IT operations. Conversely, continuous testing is a philosophy whose goal is to prevent errors from finding their way into code rather than detecting and correcting errors after the fact.

Martins breaks continuous testing into four "pillars": code quality, pipeline automation, customer experience, and application quality.

Continuous testing attempts to eliminate coding errors before the application has reached the traditional testing phase. Source: Alex Martins, via DZone

Two popular approaches for ensuring code quality are code style guides and regular peer reviews. Both techniques are intended to keep code consistent across present and future dev team members to facilitate maintenance and updates. Martins points out that code quality does not automatically translate into application quality.

Even the highest-quality code will fall flat if it can't travel freely throughout the production environment. The second of the four pillars calls for the pipeline to be automated to allow network traffic to reach its destination with no manual intervention. Whenever a human gets involved, you've created an opportunity for errors to be introduced, such as someone using an incorrect library or applying the wrong configuration files.

Devising a comprehensive, unambiguous test environment requires getting business analysts, developers, testers, and operations staff together so each area understands the needs of all the other areas. Just as the test timeline becomes a test "circle," the formerly siloed departments become a single unified organization working concurrently rather than consecutively.

Traditional approaches to DevOps testing are anything but simple. Are you dealing with virtual machines? Docker containers? A resource service? Or maybe a full Mongo cluster? Now stitch these and other components together in an integrated, continuous test environment. Definitely not simple.

What you need is the ability to wrap a handful of containers together with a half dozen VMs and then throw in a separate set of resource services, all manageable as a single instance. That's the Morpheus difference in a nutshell. 

As explained on the Morpheus support site, an instance can have multiple nodes, each representing a container or a virtual machine, depending on the provisioning engine that created the instance. Other tutorials describe how to create instances using Morpheus' automatic provisioning, and how to edit, delete, and manage Morpheus instances.

All four components of the continuous testing model are represented in the Morpheus Instance Details section. The instance info screen includes the name, description, environment, status, group, version, creation date, layout, max memory and storage, and other data. The summary window shows a summary of CPU, memory, storage, IOPS, and network throughput. The deploy screen lets you easily track deployment history.

Morpheus helps automate DevOps testing and deployment by placing all necessary information in easy-to-access instance information screens. Source: Morpheus

Once your instances are configured and deployed, use Morpheus checks to ensure peak performance. Some check types are selected automatically when you provision a service or instance type, such as database checks, web checks, and message checks. You can customize checks to run custom queries, change queue sizes, or adjust severity levels and check intervals.

Finally, use Morpheus policies to make governance and auditing quick, simple, and inexpensive. Policies can be applied to all instances at the Group or Cloud level (Cloud policies will override conflicting Group policies during provisioning). Among the policy types are expiration, host name, instance name, max containers, max cores, max hosts, max memory, max storage, and max VMs.