Algorithmia founder on the promise and pitfalls of MLOps

All Transform 2021 sessions are now available on demand. Watch now.


MLops, a combination of machine learning and information technology, is at the intersection of developer operations (DevOps), data engineering and machine learning. The goal of MLOps is to get machine learning algorithms into production.

While similar to DevOps, MLOps relies on a variety of roles and skills: data scientists specializing in algorithms, math, simulation, and developer tools, and operations administrators focused on upgrades, production deployments, resource and data management, and security. Although there are significant business value to MLPs, implementation can be difficult in the absence of a robust data strategy. Kenny Daniel, Founder and CTO of Algorithm, the company behind the enterprise MLOps platform, spoke to VentureBeat about the buzz around MLOps, its benefits and its challenges.

This interview has been edited for clarity and brevity.

VentureBeat: how does MLOps work?

Ken Daniel: MLOps applies the lessons of DevOps and software engineering best practices to the world of machine learning. MLOps encompasses all the capabilities data science, product teams, and IT operations need to deploy, manage, control, and secure machine learning and other probabilistic models in production. MLOps combines the practice of AI/ML with the principles of DevOps to define an ML lifecycle that coexists with the software development lifecycle (SDLC) for a more efficient workflow and more effective results. The goal is to support the continuous integration, development and delivery of AI/ML models into production at scale.

We specifically break down MLOps into 10 core capabilities in the deployment and editing stages of the three-step ML lifecycle (development, deployment, operations). During the implementation phase of the ML lifecycle, we have:

  1. Training integration — broad language and framework support for all DS tooling.
  2. Data Services — Native data connectors for popular platforms, as well as permissions and access controls.
  3. Design registration integrated with your documents, IDEs and SCMs, with searchability and tagging so you know the origin of all your models in production.
  4. Algorithm serving and pipelining – allowing for complex assemblies of models needed to support the app – this should be hands-off maintenance.
  5. Model management — how to manage access for versioning, A/B testing, source and license management, and building history management.

In the operational phase, there are also five core capabilities:

  1. Model operations – that’s how you manage usage and performance in production, including approval process and consent check.
  2. Infrastructure management, including fully automated infrastructure, redundancy, autoscaling, on-premise, cloud, and multi-region support.
  3. Monitoring and reporting – insight into the “who, what, where, why and when” of MLOps.
  4. Governance, logging, reporting, customer statistics for internal and external compliance.
  5. Security at all stages, including data encryption, network security, SSO and proxy compliance, consent and controls.

VentureBeat: The nature of the AI ​​implementation depends on the maturity of the organization. In this case, what needs to be in place to make an organization ready for MLOps?

Daniel: MLOps becomes relevant when trying to get machine learning models into production. This usually only happens after a data science program has been set up and the projects are in full swing. But waiting for the model to be built is too late and will lead to production delays if the MLOps story isn’t resolved.

VentureBeat: What Are Common MLOps Mistakes?

Daniel: Leave the responsibility to the individual data scientists to navigate the IT/DevOps/security departments on their own. This creates a recipe for failure, where success depends on a specialized team navigating a completely different software engineering domain. We’ve seen many companies hiring teams of data scientists and machine learning engineers and giving them standalone building models. Once they’ve built a model and need to implement it and get it ready to handle production traffic, there are a few things that need to be taken care of. These are things that are considered mandatory in the modern IT environment, not just for machine learning: source code management, testing, continuous integration and delivery, monitoring, alerting, and software development lifecycle management. Being able to effectively manage many services, and many versions of those services, is especially critical in machine learning, where models can be continuously retrained and updated. That’s why it’s critical for businesses to ask “What’s our MLOps story?” to answer. and what is the organization’s process to move from data to modeling to production.

VentureBeat: what is the most common use case with MLOps?

Daniel: Large companies use us for mission-critical applications. The most common use cases we see are those that are critical to scaling complex applications to bring agility, accuracy, or speed to market; anywhere where a faster transaction has a significant impact on value. Merck, eg. accelerates the analysis of complex compounds for drug discovery and vaccine development. EY accelerates fraud detection by updating models more often and reducing false positives by more than 30% with those better performing models. Raytheon will support development of the US Army’s Tactical Intelligence Targeting Access Node program.

VentureBeat: How has the advent of low-code/no-code MLOps helped/hindered?

Daniel: I’m generally skeptical about low/no code solutions. The good thing is that because they are typically opinionated about the applications they produce, they often come with a solid MLOps story out of the box. The downside is that while they may be quick to get started with a simple demo, most real-world applications have a complexity beyond what tools can support without code. The customization becomes crucial for applications in production.

VentureBeat: DevOps moved quickly to DevSecOps as developers realized that we needed to integrate security activities into development as well. Is there a security element for MLOps?

In our ResearchSecurity, along with governance, is the biggest challenge organizations face when deploying ML models in production. There is definitely a security element to MLOps and it converges with more traditional data and network security. Enterprise-level security is definitely something ML engineers should consider as a first-order capability of any MLOps domain. I’m talking about data encryption at rest and in flight, unique model embedding, API links, private and public CA, proxy support, SSO integration, key management, and possibly air-gapped deployment support for high-security use.

VentureBeat

VentureBeat’s mission is to be a digital city square for tech decision makers to learn about transformative technology and transactions. Our site provides essential information on data technologies and strategies to guide you in leading your organizations. We invite you to join our community to access:

  • up-to-date information on the topics that interest you
  • our newsletters
  • gated thought-leader content and discounted entry to our valued events, such as: Transform 2021: Learn more
  • network features and more

Join

Leave a Comment

x