Augtera Networks Uses AI for Intelligent, Real-Time Mining of Network Logs

Augtera Networks continues its push to offer technologies that can use unstructured text-based log data for rapid, actionable insights. IDN looks at how LogAI supports real-time data at high volumes.

by Tom Donoghue

Tags: AIOps, Augtera Networks, automation, data, logs, network operations, real-time,

Augtera Networks is pushing the envelope for Network AIOps with capabilities to automate real-time mining of network logs.  

 

The company’s LogAI is purpose-built for network operations, with the ability to process in real-time, streaming log data, from any log, without latency or message drops. LogAI builds on Augtera Networks’ expertise in Network AIOps and general AI, according to company execs.

 

“Earlier this year we announced our industry-first real-time Zero Day Anomalies technology for Syslog based on purpose-built, high-performance, Natural Language Processing (NLP),” said Rahul Aggarwal, Founder and CEO of Augtera Networks. 

 

With the recent launch of Augtera Networks’ “expanded LogAI solution” all types and formats of logs can be ingested into the Augtera platform as JSON over Kafka or using APIs.

 

As a result of this focus, LogAI can specifically address the challenges of producing actionable insights from unstructured text data at such high volumes, Aggarwal noted. These situations are key where noise levels are extremely high and “unknown unknowns” are prevalent and undetected.

 

“The semantic understanding of log messages in real-time is much more powerful than simple text searches and matching,” said Bhupesh Kothari, Augtera Networks Co-Founder and VP of Engineering. “We have expanded our platform to apply our Network AI technology including our purpose-built real-time Natural Language Processing innovation across multiple Network Operations logs.”

 

In specific, Augtera Networks' full LogAI solution, for any log format includes:

  • Support for an expanding number of log formats/transports including Syslog, JSON, JSON over Kafka, and an Augtera API.
  • Zero Day Anomaly detection of “unknown unknown” new and rare syslog messages that often precede outages.
  • Collective Learning across the entire Augtera Networks customer base of high-fidelity log classifiers.
  • Purpose-built rate change ML algorithms that identify log message bursts with high fidelity and low false positives
  • Extraction of metrics embedded in log messages with the ability to apply metric algorithms for anomaly detection and other Network AI capabilities.
  • Tag-based structured log search
  • Noiseless integration into ServiceNow, Slack, Automation, and other upstream systems.

LogAI Pushes the Envelope on Log Value for Network Operations 

Logs are a rich source of network telemetry information for network operations.

 

And while there are several tools that exist for historical analysis and querying logs, these tools have not yet provided real-time anomaly detection capabilities that can leverage the richness of text-based logs, according to the company.

LogAI was developed to address Network Operations (and Network AIOps) use cases. To process in real-time, streaming log data, from any log, without latency or message drops. 

 

In a blog post, Augtera Networks’ Mark Seery, explained the problem set the company sets out to address.

It’s no secret that one of the biggest problems plaguing network operations teams is the number of generated alerts / trouble tickets from an average of 4-10 networking tools per team. 

 

Network AIOps is a game changer. One source of false positive reduced truth, across all data sources, of what is an anomaly, what is a redundant alert / trouble ticket, and what is the lowest layer incident root object. ML algorithms that produce less false positives. Multi-layer autocorrelation that reduces redundant alarms / tickets, and better models’ relationships across all layers, to identify incident root.

The Augtera Networks website describes how LogAI supports “flexible ingestion,” as the company calls it.  

Syslog has long been the standard of network equipment vendors, however, with the rise of cloud-based systems and message streaming technologies new log formats, such as encoding logs in JSON are becoming more common. 

 

However, logs are generated by cloud-based systems as well as equipment vendors in many other non-standard formats, typically encoded as JSON. Further logs are collected, normalized, and distributed by operations teams in many other formats, typically also encoded as JSON. Kafka is an emerging message bus for many types of data within Network Operations environments.

 

LogAI supports all these scenarios today and at its core is agnostic to the format of the log messages when they are ingested, as all logs are normalized to a common internal format.

The Augtera Networks approach means leveraging LogAI allows users to see “unknown unknowns.” As the website explains:

Log messages across the industry are increasing in volume, velocity, and variety. No two equipment vendors generate the same log messages and log messages change and evolve even within the same equipment vendor, over time. When the symptoms of emerging incidents arise, they are easily missed. In addition, many persistent issues remain, lost in the noise.

“No other log solution provides this real-time NLP-based capability, and LogAI does it at high scale, with high performance, and high efficiency,” Aggarwal added.  

 

Augtera Networks solutions are being used by hyperscale cloud platforms, financial institutions, communications service providers, managed service providers, and enterprises in multiple verticals.

 




back