A Must to Stay Competitive in the Digital Space
Technology has become increasingly important in the way we work and interact. As people rely on technology in their daily lives, they develop higher expectations for functionality, usability, and performance. The life sciences industry is no different—technology is at the heart of research and development, customer interactions, and manufacturing operations.
Properly executed Agile software methodologies have become the industry standard for developing digital experiences. The correct use of Agile will drive closer alignment between strategic initiatives and business outcomes; ultimately, delivering more value to the organization. Agile can deliver value sooner, increase transparency, identify risks earlier in the process, and support adaptability.
In this white paper, we will challenge the standard life sciences industry software development methodologies and argue that by changing the timing of software validation activities, the enterprise can achieve the benefits of agility while maintaining FDA compliance.
Software validation has one goal: “Verify, Intended Use”. “We believe we should consider the least burdensome approach in all areas of medical device regulation” – FDA.1 Does that sound like your validation process?
Industry Validation Standards are Based on Old Methodologies and the Perception of Reduced Risk
Computer systems used to create, modify, and maintain electronic records to manage electronic signatures are subject to validation requirements. (See 21 CFR §11.10(a).) Such computer systems must be validated to ensure accuracy, reliability of consistent intended performance, and the ability to discern invalid or altered records. The majority of pharmaceutical companies have often followed two popular software development models: the V-Model and Waterfall. We’ll briefly describe these two models prior to discussing an Agile approach to software development and validation.
The V-Model: The V-Model, which is very similar to Waterfall, maps each phase in requirements to a corresponding phase in validation and verification that strictly follows the documentation process throughout the life of the strategic initiative. In this model, there are two concurrent streams. In stream one (the specification stream), documentation of all specifications needs must be taken, such as user, functional, and design requirements. In stream two (the testing stream), all qualifications will be performed (IQ, OQ, PQ) to support the specifications and prove intended use.
The Waterfall Model: The Waterfall model, which we’ve seen many times and possibly the most popular model in most pharmaceutical companies, requires the performance of each step in sequence—with almost zero flexibility for ad-hoc changes. The Waterfall model follows the flow of system requirements, design, development, integration and verification, installation, and validation. At the end of each phase, a complete description must be written, theoretically it would be assumed that this would make the strategic initiative simpler to be within FDA compliance. However, this flow only works in one direction and doesn’t allow flexibility to uncover potential issues and resolve as they arise.
The main benefit of following this methodology is that developers will not have any downtime, as all requirements have been defined and approved during the requirements phase. However, it’s extremely difficult, if not impossible, to define a complete list of requirements prior to coding. Many times, teams will find that upon completion of the software requirements, they are no longer valid and need to be modified. Even worse, they may need new requirements as the business need could have been misinterpreted or changed.
Why Product-Focused Agile? Deliver More Business Value
Agile allows continuous alignment between business objectives and strategic initiatives, resulting in more valued delivered:
- Quicker Realized Business Value: Delivering a working product faster shows business value even sooner. The traditional approach requires deep, long-term planning, with value delivered at the end.
- Heightened Transparency: By demonstrating working software iteratively, and allowing full access to the product backlog, there are no secrets. In a traditional model, clients have visibility during requirements and testing, but the middle part is a black box.
- Earlier Risk Identification: Risk drops dramatically over the early sprints, as the velocity of the team becomes clear. In a traditional model, the risk drops most dramatically in the final hours of the strategic initiatives—when it’s most likely unchangeable.
- Stronger Adaptability: Because the software comes in bursts of functionality, every two weeks, clients have flexibility to change throughout development. The basis of the Waterfall model is resistant to change, and the further along in the process it is, the less flexibility there is to change.
What is Agile? It’s So Much More Than Scrum
Maven Wave’s Agile approach incorporates Scrum, focusing on an inspect and adapt mindset, team orientation and product alignment supported by architecture, test-driven development, and development operations. Our Agile approach is more rigorous and lower risk than most Waterfall methodologies:
- Team Orientation: The whole team is accountable and incented to achieve the same goal.
- Servant Leadership: The organization managers and leaders empower the team to achieve goals instead of focusing on specific roles and responsibilities.
- Products Over Projects: Iteratively deploy features and measure success against business goals instead of scope and timeline (Product Approach White Paper, Optimizing Your Features).
- Solid Principles: Modular architecture that will reduce the risk of frequent change.
- User Story Discipline: Detailed “vertical slice stories” using invest standards.
- Iterative Development Cycles: Creating releasable software that can be demoed to the end user in short cycles (usually 2 weeks).
- Robust and Disciplined Framework:
- Sprint Planning – In the week prior to the start of any sprint, the product owner works with strategic initiative stakeholders to re-prioritize all stories and determine what will be worked on in the coming sprint.
- Daily Standups – The entire team (including business stakeholders that can/wish to attend) conducts daily 15 minute meetings to report status on completion of stories in the current sprint.
- Sprint Reviews – At the end of each sprint, a production-ready demonstration of completed stories is conducted by the Maven Wave team to anyone that is interested in the strategic initiatives.
- Retrospective Sessions – At the end of each sprint, the core team will meet and discuss ways to improve their work routines and product quality; this is the key differentiator between an Agile-Scrum.
- Comprehensive and Predictive Metrics: Maven Wave has built-in, quantifiable metrics within its Agile-Scrum process. These metrics are collected and tracked closely in order to empower the team, as well as management, with information necessary to identify strategic initiative impediments and make accurate strategic initiative decisions.
- Burn-Down Chart – Useful for showing how the team is progressing toward the target at a release or strategic initiative level using a quantifiable metric (story points remaining).
- Velocity Chart – How much work the team can complete during a sprint, which includes testing each function and verifying each is “production-ready”.
- Feature Parking Lot Chart – Provides a visual indicator of how close a particular feature is to completion, useful for making roadmapping and product release decision.
- Testing: Maven Wave believes in automated testing mechanisms in order to keep the team’s agility high and continue with strong throughput. Therefore, automated test suites at the unit, integration, and regression levels will be employed throughout the duration of the strategic initiative.
- Development Operations: Where possible, automate the management of environments and deployments (Dev Ops Part 1, Dev Ops Part 2).
The approach is fast, flexible, and iterative. New stories are descriptions of how members will use the services that have been included in the product roadmap and driven through the product management process. Each two week delivery cycle (a “sprint”), completes a certain number of stories that can be demonstrated with live software. This transparent approach to development ensures active engagement of business stakeholders in the process.
The size of the functionality being delivered is measured and managed, allowing a high degree of control over the cost and delivery of the solution.
Validation is Possible in the Agile Environment
The primary difference between validation in Waterfall and Agile is timing. Documentation is maintained during each sprint and formally approved prior to deployment into production. This approach allows the team to meet the necessary documentation requirements, while maintaining the flexibility to adapt as needed.
Focus on Changing the Timing, Not the Deliverables
It is a common misperception that Agile means less or even no documentation. Agile encourages the “right amount” of documentation and in a validated environment, documentation is still important. Creating and updating detailed documentation in short cycles with less unknowns will result in higher quality validation documents.
In Agile, verification and validation begin at the story level and provides traceability from user story to code, code to unit test, and user story to acceptance test. Final verification and validation is handled at the end of each sprint to ensure proof of intended use that is traced back to URS/FRS. Now, validation has become a key component to the strategic initiative as we’re validating intended use constantly and ensuring requirements match functionality. This provides a much more robust lifecycle by allowing us to identify defects as they arise and provides us the ability to resolve in upcoming sprints. See below for a sample of how documentation can be created within the Agile environment:
- Sprint 0: Architecture and Planning Sprint – The architecture and planning sprint(s) allows the team to prepare for development.
- Integrated Risk Assessment: Strategic initiative risk will be assessed to determine validation approach.
- Validation Plan & Test Plan: Created to align entire team on timing and expectations of deliverables.
- Draft URS and FRS: Known epics and features will be defined to create the foundation for the URS and FRS.
- Draft System Design Specification: Initial architecture is defined.
- IQ: Setup development environment and automate as much as possible.
- Unit Testing/Automation Testing: Set-up unit testing and automation testing framework.
- Timeline: Use relative pointing to define expected deployment dates.
- Sprints 1-N: Development Sprints (Non-Production Deployment) – Development sprints will create deployable software that can be reviewed and tested. But, in most instances there will not be deployment to production after each sprint, due to change management needs.
- Update URS and FRS: Update based on User Story details and exit criteria.
- Update System Design Specification: Update based on technical tasks defined in each story.
- Unit Testing/Automation Testing: Based on the test plan a minimum threshold for automated testing should be maintained at each sprint.
- IQ: Maintain deployment automation and update document as necessary.
- Create unit test and automation tests
- Update document-based testing tasks on stories
- Traceability Matrix: All requirements, test scripts, and expected outcomes will be tied to a story.
- User Acceptance Testing: End user or product owner will test each story and approve within the sprint.
- Draft Validation Report: Update validation report with all executed test scripts and review informally to reduce formal review timeline before production deployment.
- Identify and Track Required SOP Updates: Maintain draft SOP that can be modified if needed during each sprint.
- Update Training Materials: Create training content for each new feature.
- Tracking & Reporting: Use burndown and velocity charts to accurately predict deployment dates against targets. This method will provide significantly more accuracy than red, yellow, or green status.
- Production Deployment (ad-hoc): In preparation for a production deployment, the draft documentation will need to be approved. This approval cycle will be short if the documentation is updated and reviewed during each sprint. Production deployment can be managed as a parallel activity development sprints using Kanban or similar process.
- Approved URS and FRS: Circulate the draft URS and FRS for approval.
- Approved System Design Specification: Circulate the draft system design specification for approval.
- Approved Validation Report: Create a validation report based on the completed testing.
- Perform User Training: Perform any necessary user training, if smaller, more frequent training, is being used.
- Update and Approve SOP: Approve and publish draft SOPs.
- Other Activities: We recognize that not all activities in a large enterprise strategic initiative or program can fit into a short development cycle, but that does not mean Agile should be abandoned. These activities can be managed through Kanban methodologies. Based on our experience, these are some activities that are managed outside the iterative development process.
- Select change management and communication activities: In some instances, communications, especially during program initiation, are independent of the development cycle.
- Large-scale training programs: If a program requires large training program, such as a new core system that will significantly impact end users, some core training will probably need to be developed independent of the development cycle.
- Large scale SOP changes: Significant changes to standard operating procedures may have to be separated from software development.
- Legal and compliance activities: Legal and compliance reviews and approvals often take longer than the ideal development cycle times.
Agile has proven to be a very robust process for software development. The core principles of Agile can be adopted into most strategic initiatives to drive better outcomes. By changing the timing of validation activities, the enterprise can achieve the benefits of Agile and lower risk.
About the Authors
Michael Messeih is a Manager at Maven Wave, focused in the life sciences industry and currently engaged as a program manager for business critical validated applications.
Nicholas Polachek is a Principal at Maven Wave focused on digital strategy and transformation. He is currently the account lead at our largest life sciences client.
1 General Principles of Software Validation; Final Guidance for Industry and FDA Staff Document issued on: January 11, 2002