You are here

You are here

The top 10 criteria for evaluating AIOps platforms

public://pictures/dennis_drogseth.jpg
Dennis Drogseth Vice President of Research, IT Megatrends, Analytics and CMDB Systems, Enterprise Management Associates
 

While AIOps is increasingly taking center stage in industry discussions, and for good reason, there is also a great deal of confusion about it. Part of this is because the phrase "artificial intelligence for IT operations" often leads to debates that confuse use case and value with underlying technology capabilities.

The other reason behind the confusion is the diversity of the AIOps market in terms of capabilities, focus, deployment requirements, direction, and overarching design.

As noted in EMA's newly released Radar Report: AIOps—A Guide for Investing in Innovation

"Rather than [being] a market in the strictest sense, AIOps might be more accurately described as a 'landscape' containing within it some rather startling surprises."

The report examines 17 vendors in depth, based on extensive questionnaires and 31 deployment interviews, and provides the groundwork for the top 10 criteria. Here they are.

1. Your key requirements

EMA examined all 17 vendors across three use cases: incident, performance, and availability management; change impact and capacity optimization; and business impact and IT-to-business alignment.

We saw striking differences in both focus and design relevant to these and related value propositions, such as integrated support for DevOps or SecOps. So the first thing to do is to understand and document your own unique top priorities before you go looking.

2. Heuristic range

While this could be a data scientist's extravaganza, EMA evaluated this in large part based on four outcome-driven capabilities: anomaly detection, predictive insights, prescriptive insights (in which recommended actions are defined), and if/then analytics (if you make this change, then what will happen?).

By focusing on vendor strengths in each area, as they match your needs and readiness to go forward, you'll have an excellent technical road map for AIOps selection.

3. Integrated automation

EMA has seen the handshake between analytics and automation as perhaps the single most transformative technology investment an IT organization can make. AIOps-related automation can be provided by the vendor, or via fully supported third-party sources, or through a mixture of both.

This automation can range from workflow to IT process automation to configuration automation to workload automation—as just four key examples.

4. Required data and how it's managed

Not surprisingly, data is key for AIOps platforms to do their job well. This can typically include event, time-series data, log files, and other data such as configuration, topology, business-impact data, and even asset- and cost-related data.

5. What's already deployed

Part of understanding your requirements involves getting a firm handle on what you already have deployed—what's working well and the biggest gaps. Most AIOps platforms are designed to assimilate third-party tool sets and bring that data together, while supplementing with data collection capabilities of their own.

6. Breadth of third-party integrations

Across all 17 vendors, we saw a wide range in the number of supported integrations for third-party tool sets, from fewer than 10 to more than 100. These integrations typically include monitoring tools for infrastructure and application performance, automation options, IT service management (ITSM), and configuration management (including CMDBs), among other sources. 

7. Discovery and dependency mapping

Another capability—either directly through the AIOps platform or through integrations—is support for discovery and dependency mapping for critical contextual insights. For instance, many of the platforms also support APM (application performance management) integrations for transaction-aware insights into application-to-infrastructure interdependencies.

8. Deployment options

Deployment options can significantly affect pricing, time-to-deploy, and administrative overhead. EMA looked for flexibility in spanning both SaaS and on-premises choices.

9. Administrative overhead

EMA’s research indicated a large delta in administrative overhead for ongoing support—not including initial deployment or significant upgrades. Basic ongoing support for an enterprise-size environment (10,000 employees of 5,000 managed entities) ranged from less than 0.5 of a full-time employee to three or more.

10. Price

While software costs are critical, they were rarely the most dominant factor in selecting an AIOps platform, based on our interviews. But when costs do inevitably come into play, it's important to look at the larger context of administrative costs, deployment costs, and expected time-to-value, as well as the complexity affiliated with software licensing.

Make your AIOps investment a success

This list above provides a useful departure point for examining different AIOps platforms, but it's also important to take the time to assess your own unique world.

What are your needs? Who are the most critical stakeholders? Where and how do they agree and disagree on their priorities? What process changes are likely to be needed? Do you have top-down executive support to help your IT organization evolve to work in new and more effective ways?

All these things may be relevant to making your AIOps investment a success.

For deeper insights into what the data shows on how the AIOps market is evolving, with a fresh, in-depth look at all 17 vendors, read the EMA Radar Report: AIOps—A Guide for Investing in Innovation.

Keep learning