Exploring the Convergence of Observability and Security - Part 6: Challenges

Pete Goldin
APMdigest

With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.

Start with: Exploring the Convergence of Observability and Security - Part 1

Start with: Exploring the Convergence of Observability and Security - Part 2: Logs, Metrics and Traces

Start with: Exploring the Convergence of Observability and Security - Part 3: Tools

Start with: Exploring the Convergence of Observability and Security - Part 4: Dashboards

Start with: Exploring the Convergence of Observability and Security - Part 5: Teams

If you have already read the previous blogs in this series exploring the convergence of observability and security, the challenges will not surprise you. The experts cite compatibility of tools, teams and cultures as challenges to convergence, among others.

The following are some of the challenges experts see with achieving convergence:

Aversion to Change

Colin Fallwell, Field CTO of Sumo Logic: "Probably the biggest challenge comes down to one word. Change. Most people don't like change, much less transformation. DevSecOps requires change, it requires thinking about transformation as a continuous process that is never-ending. Up until now, this kind of transformation really could not happen, but with the rise of the Cloud Native Computing Foundation, the proliferation of open standards, and the mass adoption of OSS tooling like OpenTelemetry, and the need for proprietary agents for collecting telemetry are at an end, and with them the siloes of data."

Different Cultures

Prashant Prahlad, VP of Cloud Security Products at Datadog: "The biggest roadblock to the convergence of security and observability is culture. Security teams need to be able to trust observability teams with product security and still be able to get the visibility they need as a failsafe."

Different Priorities

Mike Loukides, VP of Emerging Tech Content at O'Reilly Media: "I think the major challenges will be the ones we've had all along. Management wants to deliver a new version on April 1. Development is under the gun to release. Ops is under the gun to deploy. And you'll still have security experts saying: Let's make sure we didn't take any shortcuts writing the code; let's make sure we're tracing the right things. It would be nice if this conflict would go away, but I don't think it will. Not now, not ever. However, putting security and ops teams in the same group will help."

Different Budgets

Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat: "The purchasing decision and budgets for observability and security may be in different organizations."

Data Silos

Buddy Brewer, Chief Product Officer at Mezmo: "Currently, many organizations unintentionally lock data in silos that only certain teams can access, which often means DevOps and SecOps teams are either not getting the right data or implementing their individual solutions to get data from the same sources. While converging security and observability will make data significantly more actionable, organizations will be met with challenges with getting the data in the correct formats to be used by different tools they may need. In addition, they must make sure that they are adhering to regulations such as GDPR and CCPA and handle personal identifiable information (PII) properly."

Tool Silos

Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) outlines several challenges to convergence. "First, the teams have separate tools with separate tool silos. Often, when these groups come together, they find the quality of the data collected by the other silo's tools are of poor quality. It's in a format that is useless to them, for instance. Also, there is no authoritative source of data. Both groups have their own data stores that represent the same truth about infrastructure and services, but the data disagrees with each other due to variations and data granularity, time stamping, etc."

"Neither group wants to give up control of tool strategy," McGillicuddy continues. "They're married to their individual tools. Which one will blink and give up their tool in favor of the other group's tool?"

Use the player or download the MP3 below to listen to EMA-APMdigest Podcast Episode 2 — Shamus McGillicuddy talks about Network Observability, the convergence of observability and security, and more.

Click here for a direct MP3 download of Episode 2 - Part 1

"We have a lot of work to do to make the tools work properly, so this is not an easy integration – largely because the observability tools were designed for observability. They were not designed for security purposes," adds Adam Hert, Director of Product at Riverbed.

Legacy Tools

Ajit Sancheti, GM, Falcon LogScale at CrowdStrike: "Legacy logging and event management tools may not provide the scale or the performance to ingest all data, which leads to ingest backlogs and sluggish search speed. Organizations should carefully evaluate logging products before attempting to collect all security and observability data in one tool."

Legacy Philosophies

Jam Leomi, Lead Security Engineer at Honeycomb: "The heart of the challenge in converging the two goes back to the culture shift we're seeing in security. A lot of today's practitioners are stuck in compliance practices or philosophies that are 30+ years old. As technology evolves, our security approach has to shift. This creates an opportunity to really connect security with the overall bottom line of the business instead of just as an afterthought. Observability as a tool and practice has the power to do a lot of the heavy lifting toward this goal, enabling a higher level of efficiency, security, and privacy."

Confidential Data

Kirsten Newcomer from Red Hat: "Some security data is not appropriate for sharing with all team members who need to consume observability data."

Security Experts are hard to find

Prashant Prahlad of Datadog: "Security experts are hard to find and take time to train within DevOps teams, so implementing DevSecOps is a long-term investment."

Knowledge Gap

Asaf Yigal, CTO of Logz.io: "Even for those that desire, or are prone to converge responsibilities, there's still a knowledge gap. Most often this is coming from the DevOps side, as in 'how do we take this important data and communicate effectively to security?' And the answer is: this is an emerging practice, so there's no wrong way, and we are working on the proverbial airplane whilst in flight!"

Despite all these challenges, Chaim Mazal, Chief Security Officer at Gigamon offers a positive outlook: "There are far fewer downsides to this convergence than there are advantages."

Go to: Exploring the Convergence of Observability and Security - Part 7: Advantages

Pete Goldin is Editor and Publisher of APMdigest

Related Links

Hot Topics

The Latest

Why AI Is No Longer Optional for IT Operations

The rise of hybrid cloud environments, the explosion of IoT devices, the proliferation of remote work, and advanced cyber threats have created a monitoring challenge that traditional approaches simply cannot meet. IT teams find themselves drowning in a sea of data, struggling to identify critical threats amidst a deluge of alerts, and often reacting to incidents long after they've begun. This is where AI and ML are leveraged ...

Outages Aren't the Enemy, Complacency Is

Three practices, chaos testing, incident retrospectives, and AIOps-driven monitoring, are transforming platform teams from reactive responders into proactive builders of resilient, self-healing systems. The evolution is not just technical; it's cultural. The modern platform engineer isn't just maintaining infrastructure. They're product owners designing for reliability, observability, and continuous improvement ...

The Changing World of Application Delivery (and why it pays to have choices)

Getting applications into the hands of those who need them quickly and securely has long been the goal of a branch of IT often referred to as End User Computing (EUC). Over recent years, the way applications (and data) have been delivered to these "users" has changed noticeably. Organizations have many more choices available to them now, and there will be more to come ... But how did we get here? Where are we going? Is this all too complicated? ...

When a Metadata Change Broke the Internet: What the Cloudflare Outage Really Revealed

On November 18, a single database permission change inside Cloudflare set off a chain of failures that rippled across the Internet. Traffic stalled. Authentication broke. Workers KV returned waves of 5xx errors as systems fell in and out of sync. For nearly three hours, one of the most resilient networks on the planet struggled under the weight of a change no one expected to matter ... Cloudflare recovered quickly, but the deeper lesson reaches far beyond this incident ...

Cybersecurity Awesomeness Podcast: Cloudflare Outage

Chris Steffen and Ken Buckler from EMA discuss the Cloudflare outage and what availability means in the technology space ...

When Milliseconds Matter: What Every Industry Can Learn From the Self-Driving Playbook

Every modern industry is confronting the same challenge: human reaction time is no longer fast enough for real-time decision environments. Across sectors, from financial services to manufacturing to cybersecurity and beyond, the stakes mirror those of autonomous vehicles — systems operating in complex, high-risk environments where milliseconds matter ...

The Path to Technojoy: How to Tackle the Silent Drivers of Technostress

Technology's role in the workplace has expanded rapidly, framing how we work and communicate. Now, with the explosion of new and innovative AI-driven tools, people are struggling to navigate how to work in this new emergent era. And although the majority of these applications are designed to make our lives easier, for many knowledge workers, they've become a source of stress and anxiety. "Technostress" ... describes the feelings of being overwhelmed by constant connectivity and cognitive overload from information and notifications, and it's on the rise ...

When Automating Forever Becomes Faster Than Doing It Once

People want to be doing more engaging work, yet their day often gets overrun by addressing urgent IT tickets. But thanks to advances in AI "vibe coding," where a user describes what they want in plain English and the AI turns it into working code, IT teams can automate ticketing workflows and offload much of that work. Password resets that used to take 5 minutes per request now get resolved automatically ...

Smarter Systems for Disinformation: How Data-Centric Design Could Transform Online Trust

Governments and social platforms face an escalating challenge: hyperrealistic synthetic media now spreads faster than legacy moderation systems can react. From pandemic-related conspiracies to manipulated election content, disinformation has moved beyond "false text" into the realm of convincing audiovisual deception ...

The 5 Pillars of Cross-Platform Observability

Traditional monitoring often stops at uptime and server health without any integrated insights. Cross-platform observability covers not just infrastructure telemetry but also client-side behavior, distributed service interactions, and the contextual data that connects them. Emerging technologies like OpenTelemetry, eBPF, and AI-driven anomaly detection have made this vision more achievable, but only if organizations ground their observability strategy in well-defined pillars. Here are the five foundational pillars of cross-platform observability that modern engineering teams should focus on for seamless platform performance ...

Exploring the Convergence of Observability and Security - Part 6: Challenges

Pete Goldin
APMdigest

With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.

Start with: Exploring the Convergence of Observability and Security - Part 1

Start with: Exploring the Convergence of Observability and Security - Part 2: Logs, Metrics and Traces

Start with: Exploring the Convergence of Observability and Security - Part 3: Tools

Start with: Exploring the Convergence of Observability and Security - Part 4: Dashboards

Start with: Exploring the Convergence of Observability and Security - Part 5: Teams

If you have already read the previous blogs in this series exploring the convergence of observability and security, the challenges will not surprise you. The experts cite compatibility of tools, teams and cultures as challenges to convergence, among others.

The following are some of the challenges experts see with achieving convergence:

Aversion to Change

Colin Fallwell, Field CTO of Sumo Logic: "Probably the biggest challenge comes down to one word. Change. Most people don't like change, much less transformation. DevSecOps requires change, it requires thinking about transformation as a continuous process that is never-ending. Up until now, this kind of transformation really could not happen, but with the rise of the Cloud Native Computing Foundation, the proliferation of open standards, and the mass adoption of OSS tooling like OpenTelemetry, and the need for proprietary agents for collecting telemetry are at an end, and with them the siloes of data."

Different Cultures

Prashant Prahlad, VP of Cloud Security Products at Datadog: "The biggest roadblock to the convergence of security and observability is culture. Security teams need to be able to trust observability teams with product security and still be able to get the visibility they need as a failsafe."

Different Priorities

Mike Loukides, VP of Emerging Tech Content at O'Reilly Media: "I think the major challenges will be the ones we've had all along. Management wants to deliver a new version on April 1. Development is under the gun to release. Ops is under the gun to deploy. And you'll still have security experts saying: Let's make sure we didn't take any shortcuts writing the code; let's make sure we're tracing the right things. It would be nice if this conflict would go away, but I don't think it will. Not now, not ever. However, putting security and ops teams in the same group will help."

Different Budgets

Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat: "The purchasing decision and budgets for observability and security may be in different organizations."

Data Silos

Buddy Brewer, Chief Product Officer at Mezmo: "Currently, many organizations unintentionally lock data in silos that only certain teams can access, which often means DevOps and SecOps teams are either not getting the right data or implementing their individual solutions to get data from the same sources. While converging security and observability will make data significantly more actionable, organizations will be met with challenges with getting the data in the correct formats to be used by different tools they may need. In addition, they must make sure that they are adhering to regulations such as GDPR and CCPA and handle personal identifiable information (PII) properly."

Tool Silos

Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) outlines several challenges to convergence. "First, the teams have separate tools with separate tool silos. Often, when these groups come together, they find the quality of the data collected by the other silo's tools are of poor quality. It's in a format that is useless to them, for instance. Also, there is no authoritative source of data. Both groups have their own data stores that represent the same truth about infrastructure and services, but the data disagrees with each other due to variations and data granularity, time stamping, etc."

"Neither group wants to give up control of tool strategy," McGillicuddy continues. "They're married to their individual tools. Which one will blink and give up their tool in favor of the other group's tool?"

Use the player or download the MP3 below to listen to EMA-APMdigest Podcast Episode 2 — Shamus McGillicuddy talks about Network Observability, the convergence of observability and security, and more.

Click here for a direct MP3 download of Episode 2 - Part 1

"We have a lot of work to do to make the tools work properly, so this is not an easy integration – largely because the observability tools were designed for observability. They were not designed for security purposes," adds Adam Hert, Director of Product at Riverbed.

Legacy Tools

Ajit Sancheti, GM, Falcon LogScale at CrowdStrike: "Legacy logging and event management tools may not provide the scale or the performance to ingest all data, which leads to ingest backlogs and sluggish search speed. Organizations should carefully evaluate logging products before attempting to collect all security and observability data in one tool."

Legacy Philosophies

Jam Leomi, Lead Security Engineer at Honeycomb: "The heart of the challenge in converging the two goes back to the culture shift we're seeing in security. A lot of today's practitioners are stuck in compliance practices or philosophies that are 30+ years old. As technology evolves, our security approach has to shift. This creates an opportunity to really connect security with the overall bottom line of the business instead of just as an afterthought. Observability as a tool and practice has the power to do a lot of the heavy lifting toward this goal, enabling a higher level of efficiency, security, and privacy."

Confidential Data

Kirsten Newcomer from Red Hat: "Some security data is not appropriate for sharing with all team members who need to consume observability data."

Security Experts are hard to find

Prashant Prahlad of Datadog: "Security experts are hard to find and take time to train within DevOps teams, so implementing DevSecOps is a long-term investment."

Knowledge Gap

Asaf Yigal, CTO of Logz.io: "Even for those that desire, or are prone to converge responsibilities, there's still a knowledge gap. Most often this is coming from the DevOps side, as in 'how do we take this important data and communicate effectively to security?' And the answer is: this is an emerging practice, so there's no wrong way, and we are working on the proverbial airplane whilst in flight!"

Despite all these challenges, Chaim Mazal, Chief Security Officer at Gigamon offers a positive outlook: "There are far fewer downsides to this convergence than there are advantages."

Go to: Exploring the Convergence of Observability and Security - Part 7: Advantages

Pete Goldin is Editor and Publisher of APMdigest

Related Links

Hot Topics

The Latest

Why AI Is No Longer Optional for IT Operations

The rise of hybrid cloud environments, the explosion of IoT devices, the proliferation of remote work, and advanced cyber threats have created a monitoring challenge that traditional approaches simply cannot meet. IT teams find themselves drowning in a sea of data, struggling to identify critical threats amidst a deluge of alerts, and often reacting to incidents long after they've begun. This is where AI and ML are leveraged ...

Outages Aren't the Enemy, Complacency Is

Three practices, chaos testing, incident retrospectives, and AIOps-driven monitoring, are transforming platform teams from reactive responders into proactive builders of resilient, self-healing systems. The evolution is not just technical; it's cultural. The modern platform engineer isn't just maintaining infrastructure. They're product owners designing for reliability, observability, and continuous improvement ...

The Changing World of Application Delivery (and why it pays to have choices)

Getting applications into the hands of those who need them quickly and securely has long been the goal of a branch of IT often referred to as End User Computing (EUC). Over recent years, the way applications (and data) have been delivered to these "users" has changed noticeably. Organizations have many more choices available to them now, and there will be more to come ... But how did we get here? Where are we going? Is this all too complicated? ...

When a Metadata Change Broke the Internet: What the Cloudflare Outage Really Revealed

On November 18, a single database permission change inside Cloudflare set off a chain of failures that rippled across the Internet. Traffic stalled. Authentication broke. Workers KV returned waves of 5xx errors as systems fell in and out of sync. For nearly three hours, one of the most resilient networks on the planet struggled under the weight of a change no one expected to matter ... Cloudflare recovered quickly, but the deeper lesson reaches far beyond this incident ...

Cybersecurity Awesomeness Podcast: Cloudflare Outage

Chris Steffen and Ken Buckler from EMA discuss the Cloudflare outage and what availability means in the technology space ...

When Milliseconds Matter: What Every Industry Can Learn From the Self-Driving Playbook

Every modern industry is confronting the same challenge: human reaction time is no longer fast enough for real-time decision environments. Across sectors, from financial services to manufacturing to cybersecurity and beyond, the stakes mirror those of autonomous vehicles — systems operating in complex, high-risk environments where milliseconds matter ...

The Path to Technojoy: How to Tackle the Silent Drivers of Technostress

Technology's role in the workplace has expanded rapidly, framing how we work and communicate. Now, with the explosion of new and innovative AI-driven tools, people are struggling to navigate how to work in this new emergent era. And although the majority of these applications are designed to make our lives easier, for many knowledge workers, they've become a source of stress and anxiety. "Technostress" ... describes the feelings of being overwhelmed by constant connectivity and cognitive overload from information and notifications, and it's on the rise ...

When Automating Forever Becomes Faster Than Doing It Once

People want to be doing more engaging work, yet their day often gets overrun by addressing urgent IT tickets. But thanks to advances in AI "vibe coding," where a user describes what they want in plain English and the AI turns it into working code, IT teams can automate ticketing workflows and offload much of that work. Password resets that used to take 5 minutes per request now get resolved automatically ...

Smarter Systems for Disinformation: How Data-Centric Design Could Transform Online Trust

Governments and social platforms face an escalating challenge: hyperrealistic synthetic media now spreads faster than legacy moderation systems can react. From pandemic-related conspiracies to manipulated election content, disinformation has moved beyond "false text" into the realm of convincing audiovisual deception ...

The 5 Pillars of Cross-Platform Observability

Traditional monitoring often stops at uptime and server health without any integrated insights. Cross-platform observability covers not just infrastructure telemetry but also client-side behavior, distributed service interactions, and the contextual data that connects them. Emerging technologies like OpenTelemetry, eBPF, and AI-driven anomaly detection have made this vision more achievable, but only if organizations ground their observability strategy in well-defined pillars. Here are the five foundational pillars of cross-platform observability that modern engineering teams should focus on for seamless platform performance ...