releases.shpreview
Datadog/Datadog Agent

Datadog Agent

$npx -y @buildinternet/releases show datadog-agent
Mon
Wed
Fri
AprMayJunJulAugSepOctNovDecJanFebMarApr
Less
More
Releases15Avg5/moVersionsv7.74.1 → v7.78.0
Jan 13, 2025

Agent

Prelude

Release on: 2025-01-13

Upgrade Notes

  • Upgraded JMXFetch to 0.49.6 which fixes a NullPointerException on JBoss when user and password not set. See 0.49.6 for more details.
  • Windows containers were updated to use OpenJDK 11.0.25+9.

New Features

  • Add metrics origins for Nvidia Nim integration.
  • APM: New configuration apm_config.obfuscation.credit_cards.keep_values (DD_APM_OBFUSCATION_CREDIT_CARDS_KEEP_VALUES) can be used to skip specific tag keys that are known to never contain credit card numbers. This is especially useful in cases where a span tag value is a number that triggers false positives from the credit card obfuscator.
  • Add new metric, container.restarts, which indicates the number of times a container has been restarted due to the restart policy. For more details: https://docs.docker.com/engine/containers/start-containers-automatically/.
  • APM: Introducing the Error Tracking Standalone config option. Only span chunks that contain errors or exception OpenTelemetry span events are taken into consideration by sampling.
  • Add new windows images for LTSC 2019 and LTSC 2022:
    • <span class="title-ref">datadog-agent:7-servercore-ltsc2019-amd64</span>
    • <span class="title-ref">datadog-agent:7-servercore-ltsc2022-amd64</span>
    • <span class="title-ref">datadog-agent:7-servercore-ltsc2019-jmx-amd64</span>
    • <span class="title-ref">datadog-agent:7-servercore-ltsc2022-jmx-amd64</span>
    • <span class="title-ref">datadog-agent:latest-servercore-ltsc2019-jmx</span>
    • <span class="title-ref">datadog-agent:latest-servercore-ltsc2022-jmx</span>
    • <span class="title-ref">datadog-agent:latest-servercore-ltsc2019</span>
    • <span class="title-ref">datadog-agent:latest-servercore-ltsc2022</span>
    • <span class="title-ref">datadog-agent:7.X.Y-ltsc2019</span>
    • <span class="title-ref">datadog-agent:7.X.Y-ltsc2022</span>
    • <span class="title-ref">datadog-agent:7.X.Y-ltsc2019-jmx</span>
    • <span class="title-ref">datadog-agent:7.X.Y-ltsc2022-jmx</span>
    • <span class="title-ref">datadog-agent:7.X.Y-servercore-ltsc2019</span>
    • <span class="title-ref">datadog-agent:7.X.Y-servercore-ltsc2022</span>
    • <span class="title-ref">datadog-agent:7.X.Y-servercore-ltsc2019-jmx</span>
    • <span class="title-ref">datadog-agent:7.X.Y-servercore-ltsc2022-jmx</span>
    • <span class="title-ref">datadog-agent:latest-ltsc2019</span>
    • <span class="title-ref">datadog-agent:latest-ltsc2022</span>
  • [ha-agent] Add haagent component used for HA Agent feature.
  • The cluster-agent now can collect pod disruption budgets from the cluster.
  • Added support for collecting container image metadata when running on a CRI-O runtime.
  • USM now monitors TLS traffic encrypted with Go TLS by default. To disable this feature, set the <span class="title-ref">service_monitoring_config.tls.go.enabled</span> configuration option to false.
  • USM now monitors traffic encrypted with Istio mTLS by default. To disable this feature, set the <span class="title-ref">service_monitoring_config.tls.istio.enabled</span> configuration option to false.
  • Introduced a new configuration variable <span class="title-ref">logs_config.http_protocol</span>, allowing users to enforce HTTP/1.1 for outgoing HTTP connections in the Datadog Agent. This provides better control over transport protocols and improves compatibility with systems that do not support HTTP/2. By default, the log agent will now attempt to use HTTP/2 (unless a proxy is configured) and fall back to the best available protocol if HTTP/2 is not supported.
  • Added a new feature flag <span class="title-ref">enable_operation_and_resource_name_logic_v2</span> in DD_APM_FEATURES. Enabling this flag modifies the logic for computing operation and resource names from OTLP spans to produce shorter, more readable names and improve alignment with OpenTelemetry specifications.
  • Add support for PHP Single Step Instrumentation in Kubernetes (not enabled by default)

Enhancement Notes

  • [ha-agent] Run HA enabled integrations only on leader Agent
  • [ha-agent] Add agent_group tag to datadog.agent.running metric
  • Cluster Agent: DatadogAgent custom resource, cluster Agent deployment, and node Agent daemonset manifests are now added to the flare archive when the Cluster Agent is deployed with the Datadog Operator (version 1.11.0+).
  • Add new host tag <span class="title-ref">provider_kind</span> from the value of <span class="title-ref">DD_PROVIDER_KIND</span> for Agents running in GCE.
  • Add query_timeout to customize the timeout for queries in the Oracle check. Previously, this was fixed at 20,000 seconds.
  • Add ability to show Agent telemetry payloads to be sent by Agent if the telemetry is enabled. One can run it with the following command: <span class="title-ref">agent diagnose show-metadata agent-telemetry</span>. See <span class="title-ref">docs <https://docs.datadoghq.com/data\_security/agent/#telemetry-collection></span> for more details.
  • Convert Prometheus style Counters and Histograms used in Agent telemetry from monotonically increasing to non-monotonic values (reset on each scrape). In addition de-accumulate Prometheus Histogram bucket values on each scrape.
  • Added support for more than 100 Aurora clusters in a user's account when using database autodiscovery
  • Adds some information about the SNMP autodiscovery status in the Agent status.
  • Adds a dedicated CRI-O Workloadmeta collector, enabling metadata collection for containers running on a CRI-O runtime.
  • Enables a cache for SQL and MongoDB obfuscation. This cache is enabled by default but can be disabled by setting <span class="title-ref">apm_config.obfuscation.cache.enabled</span> to <span class="title-ref">false</span>.
  • Improved logging to add visibility for latency and transport protocol
  • Add a new configuration option log_level for commands where the logger is disabled by default.
  • Adds initial Windows support for TCP probes in Network Path.
  • Query Aurora instances per cluster to allow up to 100 instances per cluster rather than 100 instances total.
  • The AWS Lambda Extension is now able to read the full 128-bit trace ID from the headers of the end-invocation HTTP request made by dd-trace or the datadog-lambda-go library.
  • Standardized cluster check tagging across all environments, allowing DD_TAGS, DD_EXTRA_TAGS, DD_CLUSTER_CHECKS_EXTRA_TAGS, and DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS to apply to all cluster check data when operating on the Cluster Agent, Node Agent, or Cluster Checks Runner.

Deprecation Notes

  • Deprecates the <span class="title-ref">apm_config.obfuscation.sql.cache</span> option in favor of <span class="title-ref">apm_config.obfuscation.cache</span>.
  • Remove deprecated config <span class="title-ref">otlp_config.metrics.instrumentation_library_metadata_as_tags</span>. Use <span class="title-ref">otlp_config.metrics.instrumentation_scope_metadata_as_tags</span> instead.
  • The remote tagger will attempt to connect to the core agent indefinitely until it is successful. The remote_tagger_timeout_seconds configuration is removed, and the timeout is no longer configurable.
  • The remote tagger for the trace-agent and security-agent is now always enabled and can not be disabled apm_config.remote_tagger, security_agent.remote_tagger, and event_monitoring_config.remote_tagger config entries are removed.

Security Notes

  • Fix CVE-2025-21613
  • Update golang.org/x/crypto to fix CVE-2024-45337.

Bug Fixes

  • Cluster Agent: Don't overwrite the LD_PRELOAD environment variable if it's already set, append the path to Datadog's injection library instead.
  • Fix an issue where the remote workloadmeta was not receiving some unset events for ECS containers, causing incorrect billing in CWS, CSPM, CSM Pro, CSM Enterprise, and DevSecOps Enterprise Containers.
  • Corrects the method call for gauges to be Set instead of Add.
  • Fix Oracle execution plan collection failures caused by an out-of-range position column, which can occur if the execution plan is excessively large.
  • Fix excessive number of rows coming from active session history.
  • OTLP ingestion: Stop prefixing <span class="title-ref">http_server_duration</span>, <span class="title-ref">http_server_request_size</span> and <span class="title-ref">http_server_response_size</span> with <span class="title-ref">otelcol</span>.
  • Fixes the issue of disabled services producing an error message in the event log on start. Now produces an informational message.
  • Change <span class="title-ref">kubernetes.memory.working_set</span> and <span class="title-ref">kubernetes.memory.usage</span> metrics to be of type gauge instead of rate.

Other Notes

  • Add metric origins for Platform Integrations: Fly.io, Kepler, Octopus Deploy, and Scaphandre.
  • Extend Agent Telemetry to start reporting logs.sender_latency metric.
  • The <span class="title-ref">enable_receive_resource_spans_v2</span> flag now defaults to true in Converged Agent. This enables the refactored version of the OTLP span receiver in trace agent, improves performance by 10%, and deprecates the following functionality:
    • No longer checks for information about the resource in HTTP headers (ContainerID, Lang, LangVersion, Interpreter, LangVendor).
    • No longer checks for resource-related values (container, env, hostname) in span attributes. This previous behavior did not follow the OTel spec.
  • Bumps the default value for <span class="title-ref">kube_cache_sync_timeout_seconds</span> from 5 to 10 seconds.
  • Added origin for new Milvus integration.

Datadog Cluster Agent

Prelude

Released on: 2025-01-10 Pinned to datadog-agent v7.61.0: CHANGELOG.

New Features

  • Implements the Kubernetes Admission Events webhooks. This new webhooks will emit Datadog Events when receving Validation Admission requests. It will track deployments operations made by non-system users. The webhook is controlled by using the <span class="title-ref">admission_controller.kubernetes_admission_events.enabled</span> setting.

Bug Fixes

  • The auto-instrumentation webhook no longer injects the default environment variables when disabled.
Dec 19, 2024

Agent

7.60.1

Prelude

Release on: 2024-12-19

Security Notes

  • Update golang.org/x/crypto to fix CVE-2024-45337.

Datadog Cluster Agent

7.60.1

Prelude

Released on: 2024-12-19 Pinned to datadog-agent v7.60.1: CHANGELOG.

Agent

Prelude

Release on: 2024-04-30

New Features

  • Support database-monitoring autodiscovery for Aurora cluster instances. Adds a new configuration listener to poll for a specific set of Aurora cluster IDs and then create a new database-monitoring supported check configuration for each endpoint. This allows for monitoring of endpoints that scale dynamically.
  • Add new core check orchestrator_ecs to collect running ECS tasks
  • APM stats now include an is_trace_root field to indicate if the stats are from the root span of a trace.
  • The cluster-agent now collects network policies from the cluster.
  • Enable 'host_benchmarks' by default when running the security-agent compliance module.
  • OTLP ingest now has a feature flag to identify top-level spans by span kind. This new logic can be enabled by adding <span class="title-ref">enable_otlp_compute_top_level_by_span_kind</span> in DD_APM_FEATURES.
    • With this new logic, root spans and spans with a server or consumer <span class="title-ref">span.kind</span> will be marked as top-level. Additionally, spans with a client or producer <span class="title-ref">span.kind</span> will have stats computed.
    • Enabling this feature flag may increase the number of spans that generate trace metrics, and may change which spans appear as top-level in Datadog.
  • Experimental: The process-agent checks (process, container, and process-discovery) can be run from the Core Agent in Linux. This feature can be toggled on by setting the <span class="title-ref">process_config.run_in_core_agent.enabled</span> flag to <span class="title-ref">true</span> in the <span class="title-ref">datadog.yaml</span> file. This feature is disabled by default.

Enhancement Notes

  • Add the container image and container lifecycle checks to the output of the Agent status command.
  • Add <span class="title-ref">kubelet_core_check_enabled</span> flag to Agent config to control whether the kubelet core check should be loaded.
  • Added LastSuccessfulTime to cronjob status payload.
  • Add a retry mechanism to Software Bill of Materials (SBOM) collection for container images. This will help to avoid intermittent failures during the collection process.
  • Add startup timestamp to the Agent metadata payload.
  • Agents are now built with Go 1.21.9.
  • Adds image repo digest string to the container payload when present
  • CWS: Add selftests report on Windows and platforms with no eBPF support.
  • CWS: Add visibility for cross container program executions on platforms with no eBPF support.
  • APM: Enable credit card obfuscation by default. There is a small chance that numbers that are similar to valid credit cards may be redacted, this feature can be disabled by using <span class="title-ref">apm_config.obfuscation.credit_cards.enabled</span>. Alternatively, it can be made more accurate through luhn checksum verification by using <span class="title-ref">apm_config.obfuscation.credit_cards.luhn</span>, however, this increases the performance penalty of this check.
  • logs_config.expected_tags_duration now works for journald logs.
  • [oracle] Adds <span class="title-ref">oracle.can_query</span> service check.
  • [oracle] Automatically fall back to deprecated Oracle integration mode if privileges are missing.
  • [oracle] Add service configuration parameter.
  • The connections check no longer relies on the process/container check as it can now fetch container data independently.
  • The performance of Remote Config has been significantly improved when large amounts of configurations are received.
  • Send ECS task lifecycle events in the container lifecycle check.
  • dbm: add new SQL obfuscation mode normalize_only to support normalizing SQL without obfuscating it. This mode is useful for customers who want to view unobfuscated SQL statements. By default, ObfuscationMode is set to obfuscate_and_normalize and every SQL statement is obfuscated and normalized.
  • USM: Handle the HTTP TRACE method.

Deprecation Notes

  • [oracle] Deprecating Oracle integration code. The functionality is fully implemented in the oracle-dbm check which is now renamed to oracle.

Bug Fixes

  • The <span class="title-ref">windows_registry</span> check can be run with the <span class="title-ref">check</span> sub-command.
  • CWS: Fix very rare event corruption.
  • Fixes issue where processes for ECS Fargate containers would sometimes not be associated with the correct container.
  • Fixed a bug in the Dual Shipping feature where events were not being emitted on endpoint recovery.
  • Fix issue with display_container_name being tagged as N/A when container_name information is available.
  • Fix a Windows process handle leak in the Process Agent, which was introduced in 6.52.0 when <span class="title-ref">process_collection</span> is enabled.
  • Fixes a bug where the tagger server did not properly handle a closed channel.
  • [oracle] Set the default for metric_prefix in custom_queries to oracle.
  • [oracle] Fix global_custom_queries bug.
  • [oracle] Adds the oracle.process.pga_maximum_memory metric for backward compatibility.
  • Stop sending systemd metrics when they are not set

Datadog Cluster Agent

Prelude

Released on: 2024-04-30 Pinned to datadog-agent v6.53.0: CHANGELOG.

New Features

  • APM library injection now works on EKS Fargate when the admission controller is configured to add an Agent sidecar in EKS Fargate.
  • Cluster Agent now supports activating Application Security Management, Code Vulnerabilities, and Software Composition Analysis via Helm charts.

Enhancement Notes

  • Add the <span class="title-ref">mutation_webhook</span> tag to <span class="title-ref">admission_webhooks.webhooks_received</span> and <span class="title-ref">admission_webhooks.response_duration</span> Cluster Agent telemetry.
  • When using the admission controller to inject an Agent sidecar on EKS Fargate, <span class="title-ref">shareProcessNamespace</span> is now set to <span class="title-ref">true</span> automatically. This is to ensure that the process collection feature works.
Dec 16, 2024

Datadog Agent

Release Notes

7.60.0

Prelude

Release on: 2024-12-16

Upgrade Notes

    • Parameter peer_tags_aggregation (a.k.a. environment variable DD_APM_PEER_TAGS_AGGREGATION) is now enabled by default. This means that aggregation of peer related tags (e.g., <span class="title-ref">peer.service</span>, <span class="title-ref">db.instance</span>, etc.) now happens in the Agent, which enables statistics for Inferred Entities. If you want to disable this feature, set <span class="title-ref">peer_tags_aggregation</span> to <span class="title-ref">false</span> in your Agent configuration.

    • Parameter compute_stats_by_span_kind (a.k.a. environment variable DD_APM_COMPUTE_STATS_BY_SPAN_KIND) is now enabled by default. This means spans with an eligible <span class="title-ref">span.kind</span> will have stats computed. If disabled, only top-level and measured spans will have stats computed. If you want to disable this feature, set <span class="title-ref">compute_stats_by_span_kind</span> to <span class="title-ref">false</span> in your Agent configuration.

      Note: When using peer_tags_aggregation and compute_stats_by_span_kind, a high cardinality of peer tags or APM resources can contribute to higher CPU and memory consumption. If enabling both causes the Agent to consume too many resources, try disabling <span class="title-ref">compute_stats_by_span_kind</span> first.

    It is recommended that you update your tracing libraries according to the instructions here and set DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED (or dd.trace.remove.integration-service-names.enabled) to true.

  • Upgraded JMXFetch to 0.49.5 which adds support for UnloadedClassCount metric and IBM J9 gc metrics. See 0.49.5 for more details.

New Features

  • Inferred Service dependencies are now Generally Available (exiting Beta) and enabled by default. Inferred Services of all kinds now have trace metrics and are available in dependency maps. <span class="title-ref">apm_config.peer_tags_aggregation</span> and <span class="title-ref">apm_config.compute_stats_by_span_kind</span> both now default to <span class="title-ref">true</span> unless explicitly set to <span class="title-ref">false</span>.

  • Add <span class="title-ref">check_tag_cardinality</span> parameter config check.

    By default <span class="title-ref">check_tag_cardinality</span> is not set which doesn't change the behavior of the checks. Once it is set in pod annotaions, it overrides the cardinality value provided in the base agent configuration. Example of usage:

ad.datadoghq.com/redis.checks: |
   {
     "redisdb": {
       "check_tag_cardinality": "high",
        "instances": [
         {
           "host": "%%host%%",
           "port": "6379"
         }
       ]
     }
   } 
  • Added a new feature flag <span class="title-ref">enable_receive_resource_spans_v2</span> in DD_APM_FEATURES that gates a refactored implementation of ReceiveResourceSpans for OTLP.

Enhancement Notes

  • Added information about where the Agent sourced BTF data for eBPF to the Agent flare. When applicable, this will appear in system-probe/ebpf_btf_loader.log.
  • The Agent flare now returns NAT debug information from conntrack in the system-probe directory.
  • The flare subcommand includes a --provider-timeout option to set a timeout for each file collection (default is 10s), useful for unblocking slow flare creation.
  • This change reduces the number of DNS queries made by Network Traffic based paths in Network Path. A cache of reverse DNS lookups is used to reduce the number of DNS queries. Additionally, reverse DNS lookups are now performed only for private IPs and not for public IPs.
  • Agent flare now includes system-probe telemetry data via system-probe/system_probe_telemetry.log.
  • The MSI installer uses 7zr.exe to decompress the embedded Python.
  • On Windows, the endpoint /windows_crash_detection/check has been modified to report crashes in an asynchronous manner, to allow processing of large crash dumps without blocking or timing out. The first check will return a busy status and continue to do so until the processing is completed.

Deprecation Notes

  • Prebuilt eBPF for the network tracer system-probe module has been deprecated in favor of CO-RE and runtime compilation variants on Linux kernel versions 6+ and RHEL kernel versions 5.14+. To continue to use the prebuilt eBPF network tracer, set <span class="title-ref">system_probe_config.allow_prebuilt_fallback</span> in the system-probe config file, or set the environment variable <span class="title-ref">DD_ALLOW_PREBUILT_FALLBACK</span>, to <span class="title-ref">true</span> on these platforms.
  • The feature flag <span class="title-ref">service_monitoring_config.enable_http_stats_by_status_code</span> was deprecated and removed. No impact on USM's behavior.

Bug Fixes

  • Fixes an issue added in 7.50 that causes the Windows event log tailer to drop events if it cannot open their publisher metadata.
  • Fix a bug in the config parser that broke ignored_ip_addresses from working in NDM Autodiscovery.
  • Fixes host tags with a configurable duration so the metric's context hash doesn't change, preventing the aggregator from mistaking it as a new metric.
  • Fix <span class="title-ref">could not parse voltage fields</span> error in Nvidia Jetson integration when tegrastats output contains mW units.
  • Fix building of Python extension containing native code.
  • [oracle] Fix broken activity sampling with an external Oracle client.
  • Fix nil pointer error on Oracle DBM query when the check's connection is lost before SELECT statement executes.
  • Fix a regression that caused the Agent to not be able to run if its capabilities had been modified with the <span class="title-ref">setcap</span> command.
  • Fix bug wherein single line truncated logs ended with whitespace characters were not being tagged as truncated. Fix issue with the truncation message occasionally causing subsequent logs to think they were truncated when they were not (single line logs only).

Datadog Cluster Agent

Release Notes

7.60.0

Prelude

Released on: 2024-12-16 Pinned to datadog-agent v7.60.0: CHANGELOG.

Bug Fixes

  • Fixes bug where incorrect timestamp would be used for unbundled Kubernetes events.
  • Fixed an issue in the KSM check when it's configured with the option pod_collection_mode set to node_kubelet. Previously, the check could fail to start if there was a timeout while contacting the API server. This issue has now been resolved.
Dec 3, 2024

Prelude

Release on: 2024-12-02

Enhancement Notes

  • Setting up a temporary directory for JMXFetch to use when it runs. Using the same one the Agent uses when running as this guarantees a directory where JMXFetch can write to. This helps when JMXFetch sends metrics over <span class="title-ref">Unix Domain Socket <https://docs.datadoghq.com/developers/dogstatsd/unix_socket/?tab=host></span> as it needs access to a temp directory which it can write to.
Nov 7, 2024

Agent

Prelude

Release on: 2024-11-07

Upgrade Notes

  • Removed the deprecated config option otlp_config.debug.loglevel in favor of otlp_config.debug.verbosity:
    • loglevel: debug maps to verbosity: detailed
    • loglevel: info maps to verbosity: normal
    • loglevel: warn/error maps to verbosity: basic
    • loglevel: disabled maps to verbosity: none

New Features

  • Add ability to run process/container collection on the core Agent (Linux only). This is controlled by the <span class="title-ref">process_config.run_in_core_agent.enabled</span> option in datadog.yaml.
  • DBM: Add configuration options to SQL obfuscator to customize the obfuscation of SQL statements:
    • KeepJSONPath - option to control whether JSON paths following JSON operators in SQL statements should be obfuscated. This option is only valid when ObfuscationMode is obfuscate_and_normalize.
  • APM: Add new 'sqllexer' feature flag for the Trace Agent, which enables the sqllexer imprementation of the SQL Obfuscator.
  • Introduce new Kubernetes tag <span class="title-ref">gpu_vendor</span> for the GPU resource requested by a container.

Enhancement Notes

  • Added additional Agent telemetry metrics for the log tailer code flow: logs.bytes_sent, logs.encoded_bytes_sent, and logs.bytes_missed

  • Datadog may collect environmental, performance, and feature usage information about the Datadog Agent. This may include diagnostic logs and crash dumps of the Datadog Agent with obfuscated stack traces to support and further improve the Datadog Agent.

    More details could be found in the docs

  • APM: Updates peer tags for peer.db.system.

  • Agents are now built with Go 1.22.8.

  • While using the AWS Lambda Extension, when a Lambda Function is invoked by a [properly instrumented][1] Step Function, the Lambda Function will create its Trace and Parent IDs deterministically based on the Step Function's execution context. [1]: https://docs.datadoghq.com/serverless/step_functions/installation/?tab=custom "Install Serverless Monitoring for AWS Step Functions"

  • Updates default .NET library used for auto-instrumentation from v2 to v3

  • The system-probe selinux policy is now installed on Oracle Linux

  • Increases the default input channel, processing channel, and context store sizes for network traffic paths.

  • Adds support for file log collection from Podman rootless containers when logs_config.use_podman_logs is set to true and podman_db_path is configured.

  • Allow Python integrations to emit Agent telemetry data.

Security Notes

  • Update OpenSSL to 3.3.2 (on Linux & macOS) in order to mitigate CVE-2024-6119.

Bug Fixes

  • Fixes the default configuration template to include the Cloud Security Management configuration options.
  • Fixing a bug introduced in 7.55 where in some specific scenarios, checks associated with a deleted container or POD would keep running until the Agent is restarted.
  • Fix the forwarder health check so that it reports unhealthy when the API key is invalid.
  • Fix the removal of 'non-core' integrations during Agent upgrades.
  • Fix Process Agent argument scrubbing to allow scrubbing of quoted arguments.
  • Fix Orchestrator argument scrubbing to allow scrubbing of quoted arguments.
  • Fixes an issue where TCP traceroute latency was not being calculated correctly.
  • Fixes the telemetry type for Oracle metrics.
  • APM: Fix obfuscation of SQL queries containing non-numeric prepared statement variables.

Other Notes

  • Adds Postgres integration metrics to cross-org telemetry whitelist.
  • The Agent is now built with a custom toolchain that targets our minimally supported glibc version (2.17 on x86_64 and 2.23 on aarch64)
  • On Windows, the TCP socket transport mechanism for system probe communications has been replaced with a named pipe. This deprecates the system_probe_config.sysprobe_socket configuration entry for Windows. The new fixed named pipe path is \pipedd_system_probe.
Nov 4, 2024

Prelude

Release on: 2024-11-04

Bug Fixes

  • Use of cloud-provided hostname as default when running the Agent in AKS introduced in <span class="title-ref">7.56.0</span> is reverted due to cases where the hostname returned is non-unique. This feature will be fixed and added again in a future release.
Oct 24, 2024

Agent

Prelude

Release on: 2024-10-24

Enhancement Notes

  • Removes a log statement which was causing a lot of noise in the Network Path logs.

Bug Fixes

  • [CWS] Fixes an issue where the <span class="title-ref">cws-instrumentation trace</span> command could panic before launching the traced executable when running on AWS Fargate.
  • [CWS] Fixes an issue where ECS Fargate tags would not be resolved correctly on CWS events.
  • Fixes an error in system-probe triggered by packet capture in environments with multiple VLANs.
  • Fix USM's GO-TLS support for Golang 1.23
Oct 21, 2024

Agent

Prelude

Release on: 2024-10-21

Upgrade Notes

  • Changes behavior of the timeout for Network Path. Previously, the timeout signified the total time to wait for a full traceroute to complete. Now, the timeout signifies the time to wait for each hop in the traceroute. Additionally, the default timeout has been changed to 1000ms.

New Features

  • Added capability to tag any Kubernetes resource based on labels and annotations. This feature can be configured with <span class="title-ref">kubernetes_resources_annotations_as_tags</span> and <span class="title-ref">kubernetes_resources_labels_as_tags</span>. These feature configurations are associate group resources with annotations-to-tags (or labels-to-tags) map For example, <span class="title-ref">pods</span> can be associated with an annotations-to-tags map to configure annotations as tags for pods. Example: {`pods`: {`annotationKey1`: <span class="title-ref">tag1</span>, `annotationKey2`: <span class="title-ref">tag2</span>}}
  • The Kubernetes State Metrics (KSM) check can now be configured to collect pods from the Kubelet in node agents instead of collecting them from the API Server in the Cluster Agent or the Cluster check runners. This is useful in clusters with a large number of pods where emitting pod metrics from a single check instance can cause performance issues due to the large number of metrics emitted.
  • NPM - adds UDP "Packets Sent" and "Packets Received" to the network telemetry in Linux.
  • [oracle] Add the active_session_history configuration parameter to optionally ingest Oracle active session history samples instead of query sampling.
  • Added config option logs_config.tag_truncated_logs. When enabled, file logs will come with a tag truncated:true if they were truncated by the Agent.

Enhancement Notes

  • [DBM] Bump go-sqllexer to 0.0.14 to skip collecting CTE tables as SQL metadata.
  • Agents are now built with Go 1.22.7.
  • Add the ability to tag cisco-sdwan device and interface metrics with user-defined tags.
  • Add support for setting a custom log source from resource attribute or log attribute <span class="title-ref">datadog.log.source</span>.
  • The default UDP port for traceroute (port 33434) is now used for Network Traffic based paths, instead of the port detected by NPM.
  • [oracle] Add oracle_client_lib_dir config parameter.
  • [oracle] Increase tablespace check interval from 1 to 10 minutes.
  • [oracle] Don't try to fetch execution plans where plan_hash_value is 0
  • The OTLP ingest endpoint now maps the new OTel semantic convention <span class="title-ref">deployment.environment.name</span> to <span class="title-ref">env</span>
  • Prevents the use of the <span class="title-ref">process_config.run_in_core_agent.enabled</span> configuration option in unsupported environments.
  • APM: Trace payloads are now compressed with zstd by default.

Security Notes

  • Bump embedded Python version to 3.12.6 to address <span class="title-ref">CVE-2024-4030</span> and <span class="title-ref">CVE-2024-4741</span>.
  • Update cURL to 8.9.1.
  • Update OpenSSL to 3.3.2 (on Linux & macOS) in order to mitigate CVE-2024-6119.

Bug Fixes

  • Adds missing support for the logs config key to work with AD annotations V2.
  • Fix agent jmx [command] subcommands for container environments with annotations-based configs.
  • Fixed issue with openSUSE 15 RC 6 where the eBPF tracer wouldn't start due to a failed validation of the tcp_sendpage probe.
  • Fixed a rare issue where short-lived containers could cause logs to be sent with the wrong container ID.
  • Fix Windows Process Agent argument stripping to account for spaces in the executable path.
  • Fixes issue with the kubelet corecheck where <span class="title-ref">kubernetes.kubelet.volume.*</span> metrics were not properly being reported if any matching namespace exclusion filter was present.
  • OOM Kill Check now reports the cgroup name of the victim process rather than the triggering process.
  • The process agent will no longer exit prematurely when language detection is enabled or when there is a misconfiguration stemming from <span class="title-ref">process_config.run_in_core_agent.enabled</span>'s default enablement in Kubernetes.
  • Change the datadog-security-agent Windows service display name from Datadog Security Service to Datadog Security Agent for consistency with other Agent services.
  • Fix a bug preventing SNMP V3 reconnection.

Other Notes

  • Add metric origins for the Kubeflow integration.
  • Add functional tests to Oracle using a Docker service to host the database instance.
  • Adds Agent telemetry for Oracle collector.

Datadog Cluster Agent

Prelude

Released on: 2024-10-21 Pinned to datadog-agent v7.58.0: CHANGELOG.

New Features

  • Added capability to tag any Kubernetes resource based on labels and annotations. This feature can be configured with <span class="title-ref">kubernetes_resources_annotations_as_tags</span> and <span class="title-ref">kubernetes_resources_labels_as_tags</span>. These feature configurations are associate group resources with annotations-to-tags (or labels-to-tags) map For example, <span class="title-ref">deployments.apps</span> can be associated with an annotations-to-tags map to configure annotations as tags for deployments. Example: {`deployments.apps`: {`annotationKey1`: <span class="title-ref">tag1</span>, `annotationKey2`: <span class="title-ref">tag2</span>}}
  • The Kubernetes State Metrics (KSM) check can now be configured to collect pods from the Kubelet in node agents instead of collecting them from the API Server in the Cluster Agent or the Cluster check runners. This is useful in clusters with a large number of pods where emitting pod metrics from a single check instance can cause performance issues due to the large number of metrics emitted.

Enhancement Notes

  • Added a new option for the Cluster Agent ("admission_controller.inject_config.type_socket_volumes") to specify that injected volumes should be of type "Socket". This option is disabled by default. When set to true, injected pods will not start until the Agent creates the DogstatsD and trace-agent sockets. This ensures no traces or DogstatsD metrics are lost, but it can cause the pod to wait if the Agent has issues creating the sockets.

Bug Fixes

  • Fixed an issue that prevented the Kubernetes autoscaler from evicting pods injected by the Admission Controller.
Sep 24, 2024

Prelude

Release on: 2024-09-24

Enhancement Notes

  • Agents are now built with Go 1.22.7.

Bug Fixes

  • Fix OOM error with cluster agent auto instrumentation by increasing default memory request from 20Mi to 100Mi.
  • Fixes a panic caused by running the Agent on readonly filesystems. The Agent returns integration launchers and handles memory gracefully.
Sep 18, 2024

Agent

7.57.1

Prelude

Release on: 2024-09-17

Bug Fixes

  • APM: When the UDS listener cannot be created on the trace-agent, the process will log the error, instead of crashing.
  • Fixes memory leak caused by container check.

Datadog Cluster Agent

7.57.1

Prelude

Released on: 2024-09-17 Pinned to datadog-agent v7.57.1: CHANGELOG.

Sep 9, 2024

Agent

7.57.0

Known bugs

Prelude

Release on: 2024-09-09

Upgrade Notes

  • Update cURL to 8.9.1.
  • Update OpenSSL from 3.0.14 to 3.3.1 (on Linux and macOS).

New Features

  • The <span class="title-ref">agent diagnose</span> command now includes a --json option to output the results in JSON format.
  • Add <span class="title-ref">integration</span> value for device metadata.
  • APM: In order to allow for automatic instrumentation to work in Kubernetes clusters that enforce a Restricted Pod Security Standard, which require all containers to explicitly set a securityContext, an option to configure a <span class="title-ref">securityContext</span> to be used for all initContainers created by the auto instrumentation has been added. | This can be done through the DD_ADMISSION_CONTROLLER_AUTO_INSTRUMENTATION_INIT_SECURITY_CONTEXT environment value, or admission_controller.auto_instrumentation.init_security_context configuration -in both cases a json string should be supplied.
  • Adds a <span class="title-ref">kube_runtime_class</span> tag to metrics associated with Kubernetes pods and their containers.
  • Expose the Agent's get host tags function to python checks using the new <span class="title-ref">datadog_agent.get_host_tags</span> method.
  • Implement static allowlist of Kubernetes events to send by default. This feature is only enabled when filtering_enabled is set to true in the kubernetes_apiserver integration configuration.
  • Adds a new launcher to handle incoming logs from integtrations.
  • Add optional reverse DNS enrichment of private IP addresses to NDM NetFlow.
  • On Windows, the default value for the service inference feature is now enabled.

Enhancement Notes

  • Turn on Orchestrator Explorer by default in the core agent
  • Added new source_host tag to TCP/UDP logs to help users understand where their logs came from.
  • Added support to handling UDP/TCP Logs when running the containerized agent.
  • APM: Allow custom HTTP client to be provided when instantiating the trace-agent configuration. This feature is primarily intended for the OpenTelemetry exporter.
  • APM: Add default UDS listeners for traces (trace-agent) and dogstatsd (core-agent) on /var/run/datadog/apm.socket and /var/run/datadog/dsd.socket, respectively. These are used in the Single Step APM Instrumentation, improving the onboarding experience and minimizing the agent configuration.
  • For the [Inferred Service Dependencies beta](https://docs.datadoghq.com/tracing/guide/inferred-service-opt-in/?tab=java), add two new <span class="title-ref">peer.hostname</span> precursor attributes, <span class="title-ref">out.host</span> and <span class="title-ref">dns.hostname</span>. This will improve coverage of inferred services because some tracer integrations only place the peer hostname in one of those attributes.
  • APM stats for internal service overrides are now aggregated by the <span class="title-ref">_dd.base_service</span> tag only, enhancing visibility into specific base services.
  • Include spans with <span class="title-ref">span.kind=consumer</span> for aggregation of stats on peer tags.
  • IP address quantization on all peer tags is done the backend during ingestion. This change updates the Agent to apply the same IP address quantization. This reduces unnecessary aggregation that is currently done on raw IP addresses. And therefore, improves the aggregation performance of stats on peer tags.
  • APM: Add new setting to disable the HTTP receiver in the trace-agent. This setting should almost never be disabled and is only a convenience parameter for OpenTelemetry extensions. Disabling the receiver is semantically equivalent to setting the receiver_port to 0 and receiver_socket to "".
  • Agents are now built with Go 1.22.6.
  • [NDM] Adds the option to collect BGP neighbors metrics from Cisco SD-WAN.
  • [NDM] Add option to collect cloud application metrics from Cisco SD-WAN.
  • [Cisco SD-WAN] Allow enabling/disabling metrics collection.
  • Report the hostname of Kubernetes events based on the associated pod that the event relates to.
  • Introduces a parser to extract tags from integration logs and attach them to outgoing logs.
  • Implement External Data environment variable injection in the Admission Controller. Format for this new environment variable is <span class="title-ref">it-INIT_CONTAINER,cn-CONTAINER_NAME,pu-POD_UID</span>. This new variable is needed for the New Origin Detection spec. It is used for Origin Detection in case Local Data are unavailable, for example with Kata Containers and CGroups v2.
  • Upgraded JMXFetch to 0.49.3 which adds support for jsr77 j2ee statistics and custom ConnectionFactory. See 0.49.3 for more details.
  • Windows Agent Installer gives a better error message when a gMSA account is provided for ddagentuser that Windows does not recognize.
  • Uninstalling the Windows Agent MSI Installer removes specific subdirectories of the install path to help prevent data loss when PROJECTLOCATION is misconfigured to an existing directory.
  • Adds a default upper limit of 10000 to the number of network traffic paths that are captured at a single time. The user can increase or decrease this limit as needed.
  • Language detection can run on the core Agent without needing a gRPC server.
  • Add Hostname and ExtraTags to <span class="title-ref">CollectorECSTask</span>.
  • Collect SystemInfo for Pods and ECS Tasks.
  • Implement API that allows Python checks to send logs for eventual submission.
  • Users can use DD_ORCHESTRATOR_EXPLORER_CUSTOM_SENSITIVE_ANNOTATIONS_LABELS to remove sensitive annotations and labels. For example: DD_ORCHESTRATOR_EXPLORER_CUSTOM_SENSITIVE_ANNOTATIONS_LABELS="sensitive-key-1 sensitive-key-2". Keys should be separated by spaces. The agent removes any annotations and labels matching these keys.
  • Add the ability to tag interface metrics with user-defined tags.

Security Notes

  • Fix CVE-2024-41110.

Bug Fixes

  • Results of <span class="title-ref">agent config</span> did not reflect the actual runtime config for the other services. This will have other Datadog Agent services (e.g. trace-agent) running as a systemd service read the same environment variables from a text file <span class="title-ref">/etc/datadog-agent/environment</span> as the core Agent process.
  • [DBM] Bump go-sqllexer to 0.0.13 to fix a bug where the table name is incorrectly collected on PostgreSQL SELECT ONLY statement.
  • [Cisco SD-WAN] Do not collect unspecified IP addresses.
  • Fix <span class="title-ref">container.net.*</span> metrics accuracy on Linux. Currently <span class="title-ref">container.net.*</span> metrics are always emitted with high cardinality tags while the values may not represent actual container-level values but POD-level values (multiple containers in a pod) or host-level values (containers running in host network). With this bug fix, the <span class="title-ref">container.net.*</span> metrics aren't emitted for containers running in host network and a single timeseries is emitted by pods when running multiple containers. Finally, in non-Kubernetes environments, if multiple containers share the same network namespace, <span class="title-ref">container.net.*</span> metrics won't be emitted.
  • Fix duplicate logging in Process Agent component's Enabled() method.
  • Fixed bug in kubelet check when running in core agent that was causing <span class="title-ref">kubernetes.kubelet.container.log_filesystem.used_bytes</span> to be reported by the check for excluded/non-existing containers. The metric was being reported in this case without tags. This bug does not exist in the python integration version of the kubelet check.
  • Fixes a bug on Windows in the driver installation custom actions that could prevent rollback from working properly if an installation failed or was canceled.
  • Update pro-bing library to include fix for a Windows specific issue with large ICMP packets
  • [oracle] Fix wrong durations for cloud databases.
  • Stop chunking outputs in manual checks for container, process, and process_discovery checks to allow JSON unmarshaler to parse output.
  • Remove the original pod annotation on consul
  • Fix pod status for pods using native sidecars.
  • Fix a regression where the Agent would fail to start on systems with SysVinit.
  • APM: Fixes issue where the number of HTTP decoders was incorrectly set if setting GOMAXPROCS to milli-cpu values.

Other Notes

  • Add metrics origins for vLLM integration.
  • Add deprecation warnings when running process checks on the Process Agent in Linux. This change prepares for the deprecation of processes and container collection in the Process Agent, occurring in a future release.
  • Add metric origin for the AWS Neuron integration

Datadog Cluster Agent

7.57.0

Prelude

Released on: 2024-09-09 Pinned to datadog-agent v7.57.0: CHANGELOG.

New Features

  • The Cluster Agent now supports activating Continuous Profiling using Admission Controller.
  • LimitRange and StorageClass resources are now collected by the orchestrator check.

Enhancement Notes

  • The auto-instrumentation webhook (beta) uses a new injector library.

Bug Fixes

  • Fixes a rare bug where some Kubernetes events would be emitted without a timestamp and would be dropped upstream as a result.
  • Library package versions for auto-instrumentation are now set to the latest major version of the library-package instead of <span class="title-ref">latest</span>.
    • java:v1
    • dotnet:v2
    • python:v2
    • ruby:v2
    • js:v5
  • Fix APIServer error logs generated when external metrics endpoint is activated
Sep 2, 2024

Agent

7.56.2

Prelude

Release on: 2024-09-02

Bug Fixes

  • Fix issue causing GUI to fail when opening with Internet Explorer on Windows.

Datadog Cluster Agent

7.56.2

Prelude

Released on: 2024-09-02 Pinned to datadog-agent v7.56.2: CHANGELOG.

Aug 29, 2024

Agent

7.56.1

Prelude

Release on: 2024-08-29

Bug Fixes

  • Fixed a nil pointer dereference issue in the <span class="title-ref">Tailer.DidRotate</span> function that was causing the Agent to panic.

Datadog Cluster Agent

7.56.1

Prelude

Released on: 2024-08-29 Pinned to datadog-agent v7.56.1: CHANGELOG.

Aug 21, 2024

Agent

Prelude

Release on: 2024-08-16

Upgrade Notes

New Features

  • The core Agent now supports multiple configuration files in addition to the main datadog.yaml file. The -E flag can be used to specify additional configuration files to be loaded sequentially after the main datadog.yaml.
  • When DD_SERVERLESS_STREAM_LOGS is enabled, DD_EXTENSION prints collected logs like agent stream-logs.
  • Add full support of CIS Amazon Linux 2 Benchmark in CSPM.
  • Add full support of CIS Amazon Linux 2023 Benchmark in CSPM.
  • NPM - Adds the capability to track and report failed TCP connections to the Datadog backend. This feature is disabled by default.

Enhancement Notes

  • Add the <span class="title-ref">use_apiserver_cache</span> option to the <span class="title-ref">kubernetes_state_metrics</span> check to reduce the pressure on the underlying storage engine <span class="title-ref">etcd</span>. Requires Kubernetes 1.19+.

  • APM: Add obfuscation support for OpenSearch statements within span metadata. This feature works in the same way as the existing Elasticsearch one, and is enabled by default. It is configured by binding apm_config.obfuscation.opensearch.* parameters to new obfuscation environment variables. In particular, bind: apm_config.obfuscation.opensearch.enabled to DD_APM_OBFUSCATION_OPENSEARCH_ENABLED: It accepts a boolean value with default value true.

    apm_config.obfuscation.opensearch.keep_values to DD_APM_OBFUSCATION_OPENSEARCH_KEEP_VALUES It accepts a list of strings of the form ["id1", "id2"].

    apm_config.obfuscation.opensearch.obfuscate_sql_values to DD_APM_OBFUSCATION_OPENSEARCH_OBFUSCATE_SQL_VALUES It accepts a list of strings of the form ["key1", "key2"].

  • Agents are now built with Go 1.22.4.

  • Agents are now built with Go 1.22.5.

  • Bump dependency <span class="title-ref">msodbcsql18</span> to version 18.3.3.1.

  • Adds config kubernetes_events_source_detection.enabled which is false by default. When set to true, this sets the source of kubernetes events to specific integrations based on the name of the controller that emitted it. All kubernetes events will have the tag orchestrator:kubernetes. For controller names that do not match any of the known integrations, the source will still be set to kubernetes by default.

  • Introduces a bundle_unspecifed_events config to the docker integration. When bundle_unspecifed_events and unbundle_events are true, Docker events are unbundled according to collected_event_types and the remaining events are bundled after excluding the filtered_event_types and collected_event_types.

  • The Agent will now ignore empty configuration files in conf.d. Users can use this functionality to avoid creating broken integrations when deploying agents with provisioning systems that do not allow skipping files entirely.

  • Introduces an bundle_unspecified_events config to the kubernetes_apiserver integration. When bundle_unspecified_events and unbundle_events are true, Kubernetes events are unbundled according to collected_event_types and the remaining events are bundled.

  • Improve utility functions that start or stop a service to better manage the SERVICE_START_PENDING and SERVICE_STOP_PENDING states by waiting for the service transition to complete. This will improve handling of concurrent execution of <span class="title-ref">agent.exe start</span> and <span class="title-ref">agent.exe restart</span> commands for the Windows Agent.

  • Make the <span class="title-ref">oom_kill</span> check capture the OOM score and the OOM score adjustment of the process being killed.

  • Oracle integration will now auto-adjust the size of the SQL substring requested by the activity sampler to better support users with multi-byte character sets, for example, Korean, Arabic, etc. This should alleviate crashes caused by long queries using these characters.

  • The OTLP ingestion endpoint now supports the same settings and protocol as the OpenTelemetry Collector OTLP receiver v0.103.0.

  • APM: Probabilistic Sampler now only looks at the lower 64 bits of a trace ID by default to improve compatibility in distributed systems where some apps may truncate the trace ID. To maintain the previous behavior use the feature flag <span class="title-ref">probabilistic_sampler_full_trace_id</span>.

  • Adds the source of the payload for Processes-owned messages.

  • Add tags to <span class="title-ref">CollectorManifest</span>

  • Add image and imageID fields to pods ContainerStatuses.

  • The orchestrator check can now scrub sensitive data from probes in pods specifications.

  • The Agent now populates the <span class="title-ref">git.repository_url</span> and <span class="title-ref">git.commit.sha</span> tags from the values of the <span class="title-ref">DD_GIT_REPOSITORY_URL</span> and <span class="title-ref">DD_GIT_COMMIT_SHA</span> container environment variables.

  • Implement the kubernetes_persistent_volume_claims_as_tags configuration that allows users to disable PersistentVolumeClaim for Kubernetes resources.

  • Upgrade the NTP check client library 'beevik/ntp' from v0.3.0 to v1.3.4

  • Use cloud-provided hostname as default when running the Agent in AKS.

  • APM: Enabled zstd compression by default on trace payloads.

Deprecation Notes

  • APM: DD_APM_MAX_TPS config setting is deprecated in favor of the more accurate DD_APM_TARGET_TPS. Accordingly, when configured through YAML, max_traces_per_second is deprecated in favor of target_traces_per_second. The setting behavior remains the same, only the name is changed to more accurately reflect the existing logic.

Security Notes

  • Updating OpenSSL to 3.0.14 to address CVE-2024-4741 (on Linux and macOS).

Bug Fixes

  • Upgrades the pro-bing library to fix a Windows-only bug with too-long ICMP packets being received
  • Fix ExtraTags mapping for CollectorManifest.
  • Fix a bug in the Agent where it could potentially fetch logs of short-lived Kubernetes jobs twice if the CRI is Docker.
  • Re-enable printing of checks metadata in the datadog-agent status collector section.
  • Fix OTLP status output not being displayed in the GUI.
  • Fix issue where init config for ping took priority over instance config.
  • Fix diagnose command for logs endpoints and related warnings about unknown config keys.
  • Fixes <span class="title-ref">oracle.tablespace.offline</span> metric not emitting 1 when tablespace is offline.
  • APM: Show probabilistic sampling configuration in Agent status when enabled.
  • Add a field to differentiate between empty and undefined podSelector or namespaceSelector for network policies.
  • Fixed a bug where the file tailing position is always set to the beginning, this fix allows users to explicitly set a starting position.
  • All datadog public endpoints have the maximum requirements to close idle connections after 60s being idle. If a given client keeps it for longer, the server will close it, and the client will likely see the issue during the next write, leading to a connection reset error. The idle timeout should be therefore set under a minute. This PR is reducing the timeout from 90 to 30s.
  • Windows: Added driver rollback properties to ensure that all services and drivers are uninstalled or rolled back after an installation or upgrade failure.

Other Notes

  • Add metric origins for community Python integrations.

Datadog Cluster Agent

Prelude

Released on: 2024-08-16 Pinned to datadog-agent v7.56.0: CHANGELOG.

Upgrade Notes

  • Disables default injection of the .NET profiler dependency for Kubernetes auto_instrumentation.

Enhancement Notes

  • Mark the NetworkPolicy collector as stable in the Cluster Agent
  • Enabled language detection automatically in the injected agent sidecar on EKS Fargate when APM SSI is enabled. This is only available for users using the admission controller to automatically inject the agent sidecar.
  • The orchestrator check can now scrub sensitive data from probes in pods specifications.

Bug Fixes

  • Fixes issue where the external metrics server would sometimes return metrics which had not been updated for longer than the configured <span class="title-ref">external_metrics_provider.max_age</span> as valid. In connection with this fix, a new config (<span class="title-ref">external_metrics_provider.query_validity_period</span>) has been added to account for the delay between when metrics are resolved and when they are queried by the various autoscaling controllers. It is set to 30 seconds by default.
Aug 5, 2024

Agent

7.55.3

Prelude

Release on: 2024-08-01

Enhancement Notes

  • Agents are now built with Go 1.21.12.

Security Notes

  • Fix CVE-2024-41110.

Datadog Cluster Agent

7.55.3

Prelude

Released on: 2024-08-01 Pinned to datadog-agent v7.55.3: CHANGELOG.

Jul 29, 2024

Agent

7.55.2

Prelude

Release on: 2024-07-25

Security Notes

  • Fix CVE-2024-6257.

Bug Fixes

  • Fixes an issue introduced in <span class="title-ref">7.55.0</span> with container metrics. In some rare cases, container metrics (cpu, memory, limits, etc.) could be incorrect and not reflect actual resources usage.

Datadog Cluster Agent

7.55.2

Prelude

Released on: 2024-07-25 Pinned to datadog-agent v7.55.2: CHANGELOG.

Jul 15, 2024

Agent

Prelude

Release on: 2024-07-12

Bug Fixes

  • Fix a regression where the agent would fail to start on systems with SysVinit

Datadog Cluster Agent

Prelude

Released on: 2024-07-12 Pinned to datadog-agent v7.55.1: CHANGELOG.

Jul 11, 2024

Agent

Known bug

The datadog-agent service fails to start on systems using the SysVInit init system, with a datadog-agent: unrecognized service error. See https://github.com/DataDog/datadog-agent/issues/27523 for details.

Prelude

Release on: 2024-07-11

New Features

  • Add new parameter djm_config.enabled to enable Data Jobs Monitoring
  • You can now set the buffering configuration of Lambda Logs API using DD_LOGS_CONFIG_LAMBDA_LOGS_BUFFERING_TIMEOUT_MS, DD_LOGS_CONFIG_LAMBDA_LOGS_BUFFERING_MAX_BYTES, and DD_LOGS_CONFIG_LAMBDA_LOGS_BUFFERING_MAX_ITEMS.
  • Adds the new service_discovery check, which is currently only supported on Linux.
  • APM: OTLP: Added an API in pkg/trace/stats that converts eligible OTLP spans to the input to APM stats concentrator.
  • Experimental support of SBOM generation on Windows.
  • Expose the Agent's MongoDB command string obfuscation to Python checks using the new <span class="title-ref">datadog_agent.obfuscate_mongodb_string</span> method.
  • [network path] Add feature to traceroute network connections.
  • Implement OpenTelemetry SDK resource attributes as unified service rags.
  • CSM captures and generates events based on IMDS traffic
  • Add support for <span class="title-ref">kubernetes_namespace_annotations_as_tags</span>. This new option is equivalent to the existing <span class="title-ref">kubernetes_namespace_labels_as_tags</span>, but it considers namespace annotations instead of namespace labels. With this new option, users can enrich tagging based namespace annotations.

Enhancement Notes

  • Add a section in the Agent status to show device profile parsing errors.
  • Add Go execution traces to Agent Flare.
  • Introducing the port conflict suite to the Agent diagnose command. Adding this for MacOS only first. Will incrementally add support for Linux and Windows. The Agent might need particular permissions to read information about processes using ports.
  • Agents are now built with Go 1.21.11.
  • [NDM] Add option to collect hardware status from Cisco SD-WAN.
  • Fix concurrency handling by Agent restart-service CLI command when the Agent starts and simultaneously restarts. The Agent may start and restart concurrently during the Agent's installation or upgrade.
  • OOM kill events in the OOM kill check are now reported at ERROR level instead of INFO level.
  • Improve concurrency handling for Agent start/simultaneous restart. Agent starting and restarting concurrent execution may happen during the Agent's installation or upgrade.
  • Add support for parsing <span class="title-ref">pool</span> and <span class="title-ref">peer</span> directives in locally defined configuration files in the NTP check.
  • Improve error handling in the Lambda Logs API handler not to ignore error details
  • Improve NTP check log messages.
  • Also add <span class="title-ref">kube_service</span> tag to pods that run with <span class="title-ref">hostNetwork: true</span>.
  • Added a new -l flag to the datadog-agent status command. This flag allows users to print all available sections.
  • Improve NTP check error messages to show the specific hosts that are failing.
  • Store SNMP device tags on disk to avoid losing them when the agent restarts.
  • The Datadog agent container image is now using Ubuntu 24.04 noble as the base image.
  • Improve GUI security authorization flow.

Deprecation Notes

  • Deprecated datadog-agent status component [section] in favor of datadog-agent status [section].

Security Notes

  • Current GPG keys that are used to sign new releases of the Agent package are about to expire. Following our 2024 GPG key rotation plan, we rotated RPM and APT GPG keys.

Bug Fixes

  • Fixes the validation of permissions for the secret backend helper.
  • APM Stats type for is_trace_root flag is now renamed to trilean.
  • [DBM] Bump go-sqllexer to 0.0.12 to fix a bug where table name is incorrectly collected on statement CREATE TABLE IF NOT EXISTS.
  • Correct log level of trace writer log message
  • Fix an issue introduced in Agent 7.51 for Windows that would cause a panic during Agent shutdown if the etwtracer component failed to initialize.
  • Fixed an issue where the datadog-agent status and datadog-agent flare commands could hang while trying to reach other agent processes.
  • Fix an issue where the Agent incorrectly reports JMX integrations as having issues.
  • Fix leader election error when running checks manually from the Cluster Agent.
  • [oracle] Fix missing query metrics.
  • Fix metric reporting for process checks on the core Agent.
  • Fix the status date so that it is computed for every request.
  • Fix a bug where containerd container metrics and container tags were not being collected for containers with arbitrary container IDs.

Other Notes

  • Added metrics origins for python integrations.
  • Added metrics origins for the most recent python integrations.

Datadog Cluster Agent

Released on: 2024-07-11 Pinned to datadog-agent v7.55.0: CHANGELOG.

New Features

  • Add support for <span class="title-ref">kubernetes_namespace_annotations_as_tags</span>. This new option is equivalent to the existing <span class="title-ref">kubernetes_namespace_labels_as_tags</span>, but it considers namespace annotations instead of namespace labels. With this new option, users can enrich tagging based on namespace annotations.
  • Support namespace labels as tags on kubernetes events.

Enhancement Notes

  • Add reason:backofflimitexceeded,deadlineexceeded to the kubernetes_state.job.failed metric to help users understand why a job failed.
  • Reduced the memory used to store the tags.
  • The Datadog cluster-agent container image is now using Ubuntu 24.04 noble as the base image.

Bug Fixes

  • Fixes an issue with large clusters where the Cluster Agent fails to collect all tags when <span class="title-ref">cluster_agent.collect_kubernetes_tags</span> is enabled.
Jun 19, 2024

Agent

Prelude

Release on: 2024-06-18

Datadog Cluster Agent

Prelude

Released on: 2024-06-18 Pinned to datadog-agent v7.54.1: CHANGELOG.

Latest
7.78.0
Tracking Since
Apr 20, 2023
Last checked Apr 21, 2026