OpenSearch Adds Experimental Tracing Tools
OpenSearch has introduced experimental support for distributed tracing and metrics analytics. The new tools aim to create a unified observability experience across logs, traces, and metrics, which is considered essential for debugging complex distributed systems and data pipelines.
- The new tools are designed to integrate with OpenTelemetry (OTel), a Cloud Native Computing Foundation (CNCF) project that provides a vendor-neutral, open-source standard for collecting telemetry data. - The end-to-end data pipeline involves instrumenting applications using OTel SDKs, sending data to an OpenTelemetry Collector, and then using OpenSearch Data Prepper to transform and enrich the data before indexing. - This tracing feature was released as experimental in OpenSearch 2.10 and requires users to enable a specific feature flag (`opensearch.experimental.feature.telemetry.enabled`) to begin use. - This move positions OpenSearch as a more direct competitor to established open-source tracing tools like Jaeger, which was originally developed at Uber and can also use OpenSearch as a storage backend. - OpenSearch originated as a fork of Elasticsearch and Kibana in early 2021 after their license was changed to be non-opensource. - The project's public roadmap for 2024-2025 identifies "Observability, Log Analytics, and Security Analytics" as one of the main investment themes, signaling continued development of these capabilities. - Analysis and visualization of trace data, including service maps and latency histograms, are handled within OpenSearch Dashboards. The system uses a specific query language called Piped Processing Language (PPL) for interactive data exploration.
