Is there a need to change traditional testing approaches? What is DevOps? Who’s using it? There are a couple of questions that will be addressed in this Solution Spotlight, as well as tips and pointers around reducing challenges with integration challenges.
How Continuous Development can Reduce Cost Challenge in ALM:
Application lifecycle management (ALM) provides a systematic process for managing a software development project through the requirements gathering, design, development, testing and maintenance phases. Even though ALM provides a systematic and disciplined way to manage a potentially chaotic process, it results in a number of costs. Many of them are due to the nature of ALM itself, resulting in wasted time and effort of development and testing resources.
Continuous development (or continuous integration, as it is called sometimes) provides a way of managing software development by coordinating rapid software builds with automated testing. These combined build and automated-test cycles are done multiple times a day or even an hour. Continuous development enforces a discipline of having all developers check in the changes that they have done in a day or since the last build. Automated tests are then run quickly as part of the build itself. If the automated tests fail, developers are forced to sort out the defects and integration issues immediately and add changes into the next build. This cycle is repeated quickly until all the automated tests are successful. Once a build passes all the automated tests, it is tested with the data from the production system in use at that time, and then the production system is updated if those tests are successful.
Continuous development reduces the costs of ALM in a number of ways, including the costs of delayed development and deployment cycles. Here are some of the ways continuous development addresses the costs of ALM:
- Requirements validation costs: Requirements gathering and validation is one of the pernicious costs in ALM. If requirements are not gathered properly and validated, you could spend time designing and developing the wrong solution that does not match the actual requirements. Miscommunication causes requirements rework costs. Continuous development forces the early and rapid production of working code. Requirements can be validated early on in the ALM cycle. In many software development projects, even a user interface prototype created early on with continuous development may be enough to validate requirements gathered so far. If course corrections are made at this early stage, you can avoid the costs of reworking more expensive, fully developed software.
- Design validation costs: Continuous development, with its early and rapid builds, helps development teams prioritize important and open design questions, allows them to prototype alternative designs and test them out early. Unknown questions about complex software architecture always plague design. Continuous development allows for attacking these questions early and quickly, getting them out of the way and avoiding expensive redesign of already developed software.
- Testing costs: In traditional ALM cycles, the quality assurance/testing group waits for the development build to start their testing. Now the development group waits for the testing group to finish their testing and report their defects and issues. There is a lot of waiting, resulting in waste of time and human resources. In continuous development, tests are automated and incorporated into the build itself. The testing resources spend their time in designing automated tests and incorporating them into the build process. This they can do continuously, without waiting for any other activity to happen. Developers are continuously addressing the issues automated tests bring up and resolve them all before the next build. Testing costs are radically reduced simply by the use of automated tools and elimination of waiting time in the ALM cycle.
- Costs of regression errors and rework: Regression errors creep into builds if developers are not disciplined and do not follow the prescribed process for checking in their latest code changes. If by mistake, instead of the latest version, an older version is checked in, bugs already fixed may creep back into the build. These need to be found again in testing. Regression error costs are completely avoided by the automated check-in, build and test processes that continuous development dictates. Builds are not complete if they do not pass all automated tests so far. Rework due to regression errors is totally eliminated.
- Integration costs: Integration testing is usually a separate activity in ALM. Continuous development dictates that all modules be integrated in every build. Integration tests also form part of the automated tests. This means that a build does not pass unless it goes through the full integration testing every time a build is done. There are no separate integration testing costs with continuous development and, in addition, it improves the quality of code.
- Release costs: In ALM, you have a separate release process. This results in unexpected costs due to delays and opportunity costs in bringing down your production system for maintenance and upgrades. Continuous development negates the need for a separate and discrete release cycle. When daily or hourly builds pass all automated tests with a copy of the actual data from the production system, the new build can be installed on production without any further delay. This eliminates release costs and the waiting time of development and testing resources in ALM.
Conclusion
Application lifecycle management is a disciplined process for software development with proper requirements gathering, design, development, testing and maintenance phases. However, time and effort could potentially be wasted by development and testing resources at various phases of ALM, resulting in various costs. Continuous development eliminates a lot of these costs simply by enforcing a check-in discipline with software version management tools. It also makes sure that automated tests become part of every build and so eliminates much of the manual testing necessary, speeding up the development cycle itself.
Moving to DevOps Speeds Deployment and Boosts ROI
Is there a need to change traditional testing approaches? What is DevOps? Who‘s using it? These are a couple of the questions addressed by Manoj Narayanan of Cognizant Technology Solutions at STAREAST 2012 in his session, ―Testing in the DevOps World of Continuous Delivery.‖ He explained the benefits of DevOps and how it impacts the organization.
Traditional testing approaches are geared toward sequential delivery, which leads to an emphasis on discrete skill sets and therefore minimizes cross-functional behavior. The sharing of ideas and mutually beneficial collaboration is limited as there are barriers between development, testing and deployment. Narayanan identified the drawbacks to the sequential delivery approach as a longer wait time and higher risk; essentially a developer may be saying, ―I don‘t know if what I‘m building is what the end user wants.
In order to meet the objectives of business – to reduce risk, respond fast and beat competitors – Narayanan supports moving from a discrete delivery approach to a continuous delivery approach. The language changes from ―build it right‖ to ―build the right it.
Agile development and DevOps
Agile development offers one path to continuous delivery as it brings developers and testers together and focuses on iterative development. Yet, according to Narayanan, ―the ‗last mile problem‘ still exists.‖ The wall separating deployment from the development and testing side encourages a long time between testing and deployment, and furthermore, testers do not necessarily have a systems administration skill set.
DevOps offers a potential solution to this integration challenge, bringing together development, testing and operations all on the same team, a team capable of playing all the different roles.
“Like Agile, this approach aims at getting features deployment ready at a high frequency. But where Agile primarily focuses on the functional and non-functional readiness of the application, DevOps takes it one step further and ensures the Operational and Business readiness as well,” Narayanan writes in his blog.
Narayanan noted several websites that are currently employing the DevOps model, sites like Facebook, Etsy, Orbitz, Groupon and Flickr. He explained that the frontrunners have tended to be Web 2.0 firms with strong reliance on eCommerce where fast changes and improved response time are paramount. He mentioned that Netflix is involved in this space as well, actually going with a NoOps approach.
When it comes to testing, the change to the DevOps environment must be gradual, and the talent involved will need to acquire new skills and adapt to new responsibilities. Testers need to gain knowledge in development languages— which fortunately is a bit easier with user-friendly tools like Python and Cucumber – and they also need to learn deployment processes and tools. Developers and systems administrators, on the other hand, must learn about test processes, design techniques and related tools.
Making the change to DevOps
Shifting to DevOps also makes continuous integration mandatory, according to Narayanan. He explained that in test-driven development we expect to fail first, but those errors inform the next cycle of development and testing. He emphasized that teams must have strong discipline around the single source code repository. The build process can be automated, resulting in a fast build. In order for all of these elements to work effectively, he highlighted the need for transparency; everyone on the team needs to know what is happening.
The process is also affected by the heavy reliance on innovative automation, which is embedded early in the lifecycle. ―‘Smart testing‘ is dissolving the boundaries of traditional system and integration testing,‖ said Narayanan. Teams can now ―leverage an optimal mix of automation across the lifecycle,‖ which includes automated unit testing, automated service layer testing and automated regression testing. Continuous integration is further facilitated by release management automation.
Governance, technology and stakeholder buy-in
When it comes to governance, Narayanan holds that teams work best in pods which are comprised of ‗jacks of all arts;‘ these pods are centrally coordinated and are characterized by ongoing communication and collaboration. The incremental changes taking places are driven by business goals. One result of this pod arrangement is that a greater need for specialists arises; they take on the challenging sub-tasks that the regular pod members may not be equipped to handle. Pod members can also identify repeatable tasks and automate them. This approach works especially well in smaller and mid-sized organizations.
The cultural impact of DevOps also appears in the technology the organization uses. DevOps teams need user-friendly tools that all team members—developers, testers and systems administrators— can learn and use. In addition, the focus on automation and reusability intensifies.
In order for stakeholders to buy in to these changes, communication is key, and stakeholder buy-in, likewise, is key to the success of this model. The success may be viewed in terms of speed and ROI—as the deployment happens faster, ROI improves.
