From Science to Delivery: Making the Most of Machine Learning with DevOps Principles

Machine learning is garnering a great deal of attention and resources right now but experience is showing that the journey from promising experiments to functioning solutions is not always an easy one. In order to avoid having pilots being no more than one-off science experiments it takes special care and effort. One way to deal with this problem is to adopt the principles of DevOps as useful guideposts to help with the operationalizing of machine learning projects. With this approach, there is a much greater chance of sustained success from machine learning investments.

Root of the Problem

The adoption of machine learning promises to unleash whole new levels of insight and innovation to virtually all businesses but the very attributes that make it so powerful also mean that it can pose particular challenges in development and, more importantly, in operation. Specifically, issues relating to data, diversity of resources, and fast evolving or newly created software paradigms can hinder the ability to successfully launch and operationalize machine learning initiatives.

By definition, machine learning needs to consume vast amounts of data in order to work effectively and this puts an emphasis on new forms of data acquisition as well as discipline in data management. An organization must be agile when it comes to meeting the data needs and requests generated by machine learning project teams. Facets such as lineage, versioning, movement and transformation, in addition to quantity, take on new significance and generate unique demands. Existing data management techniques and functions aren’t likely to suffice so new approaches are required.

The adoption of machine learning brings new breeds of team members into the mix and this can be challenging because most existing technical operations teams are somewhat heterogeneous. In existing teams, members share a technical background that is usually grounded in software engineering disciplines where functions and processes are well defined and understood. With machine learning projects, team members may come from other disciplines such as data science and my even include subject matter experts such as medical doctors, lawyers, or other professionals. The addition of non-technical members can create dissonance on a team as the focus on doing science comes into conflict with delivering results.

Finally, machine learning is new and developing at a very fast pace and this leads to a general lack of vocabulary to define technology paradigms and even job roles. This lack of standardization and best practices can have a practical impact in that it is hard to perform when you have yet to set norms. One result that is particularly germane to this discussion is that teams do not have the required number of specialists who can help when it comes to scaling solutions.

One particular item of note that may combine all of the above factors is model decay. Simply put, markets are dynamic and machine learning outcomes can’t be expected to be immutable and everlasting. For example, exogenous factors such as new tax laws can shift consumer behavior while the supply side might be affected by something such as the new availability of a generic drug. In either example, models will need to be retrained in order to be kept up to date with current market conditions. This can require fast and flexible data acquisition and deployment, the team resources to efficiently respond to changing conditions, and adroit learning and response cycles to roll and grow as conditions evolve.

DevOps Principles as a Template for Success

In this environment, looking to other transformative disciplines for inspiration and guidance can be extremely useful. In particular, the implementation of DevOps principles has had a profound impact on the quality, quantity, and efficiency of moving software from development into operation. The evolution of DevOps thinking is a natural progression of taking an agile mindset from isolation in development to widespread utilization as changes are operationalized. In the case of machine learning, three DevOps principles in particular seem to have particular relevance:

1. Empowering cross-functional teams with clearly elucidated business goals
2. Ensuring that there is continuous, rapid, and useful feedback
3. Automating as much of the process as is possible

Cross-Functional Teams

The concept of mobilizing cross-functional teams that are operating with clearly elucidated business goals may seem axiomatic, but it is easier said than done. Remember that these teams will be incorporating new types of members and that they will be working in conjunction with existing teams that are using processes and protocols that may not meet the needs of a machine learning project. Also, the field is new and experiencing rapid change, making deeply experienced practitioners a scarce commodity. For all of these reasons, it is all the more imperative that teams not only have the resources that they need but that business goals are clearly defined and communicated. A new level of candor and discipline will be required for teams to function smoothly and maximum results to be achieved.

Continuous, Rapid Feedback

The adoption of machine learning also puts special demands on the ability to work fast. The need for rapid cycles is inherent in machine learning because there is a constant flow of experimentation and learning that goes on as a model is trained. A great deal of time and effort goes into ensuring that data is clean and complete and, at the same time, working through the process will almost certainly reveal that new or alternative forms of data will be required. This means that not only must the organization be proficient at procuring data but they must be flexible in doing so because some of the new data is likely to require new forms and delivery. Operations can be strained – and machine learning projects slowed or compromised – by a lack of speed and flexibility.

Automation

Finally, the DevOps principle of automating with machines so that humans can improvise and innovate is relevant to machine learning as well. In the discipline of machine learning, this idea is taking the form of AutoML, which automates the selection, tuning, modeling, and assessment routines. The idea behind AutoML is that human intelligence is critical to maximizing results and these resources can be wasted in executing manual spade work when they should be devoted to tasks with higher potential rewards. The adjustment of parameters in a machine learning routine can have a virtually infinite number of permutations and manually adjusting them would be exceedingly complicated and consume far too much time. Instead, tools can be deployed that are pre-set to account for as many of these permutations as possible and to automatically carry out the routines to check all prescribed outcomes. This automation not only hastens outcomes but also allows human resource cycles to be put to better use which inevitably opens up the possibility of greater insight and business results.

Making the Most of Machine Learning Efforts

Machine learning is a powerful discipline that has the power to transform many aspects of business but utilizing it to its full potential is not without its challenges. Taking a holistic approach, moving quickly with integrated feedback, and maximizing automation wherever possible are the keys to success. At Maven Wave, we focus on the big picture of transformation and can deliver results in a discipline such as machine learning that is characterized by rapidly evolving technology, complex interaction of business areas and resources, and the potential for big outcomes for effort deployed.