Release on: 2025-01-13
NullPointerException on JBoss when user and password not set. See 0.49.6 for more details.container.restarts, which indicates the number of times a container has been restarted due to the restart policy. For more details: https://docs.docker.com/engine/containers/start-containers-automatically/.DatadogAgent custom resource, cluster Agent deployment, and node Agent daemonset manifests are now added to the flare archive when the Cluster Agent is deployed with the Datadog Operator (version 1.11.0+).query_timeout to customize the timeout for queries in the Oracle check. Previously, this was fixed at 20,000 seconds.log_level for commands where the logger is disabled by default.remote_tagger_timeout_seconds configuration is removed, and the timeout is no longer configurable.apm_config.remote_tagger, security_agent.remote_tagger, and event_monitoring_config.remote_tagger config entries are removed.golang.org/x/crypto to fix CVE-2024-45337.logs.sender_latency metric.Released on: 2025-01-10 Pinned to datadog-agent v7.61.0: CHANGELOG.
Release on: 2024-12-19
golang.org/x/crypto to fix CVE-2024-45337.Released on: 2024-12-19 Pinned to datadog-agent v7.60.1: CHANGELOG.
Release on: 2024-04-30
1.21.9.logs_config.expected_tags_duration now works for journald logs.service configuration parameter.normalize_only to support normalizing SQL without obfuscating it. This mode is useful for customers who want to view unobfuscated SQL statements. By default, ObfuscationMode is set to obfuscate_and_normalize and every SQL statement is obfuscated and normalized.oracle-dbm check which is now renamed to oracle.display_container_name being tagged as N/A when container_name information is available.metric_prefix in custom_queries to oracle.global_custom_queries bug.oracle.process.pga_maximum_memory metric for backward compatibility.systemd metrics when they are not setReleased on: 2024-04-30 Pinned to datadog-agent v6.53.0: CHANGELOG.
Release on: 2024-12-16
Parameter peer_tags_aggregation (a.k.a. environment variable DD_APM_PEER_TAGS_AGGREGATION) is now enabled by default. This means that aggregation of peer related tags (e.g., <span class="title-ref">peer.service</span>, <span class="title-ref">db.instance</span>, etc.) now happens in the Agent, which enables statistics for Inferred Entities. If you want to disable this feature, set <span class="title-ref">peer_tags_aggregation</span> to <span class="title-ref">false</span> in your Agent configuration.
Parameter compute_stats_by_span_kind (a.k.a. environment variable DD_APM_COMPUTE_STATS_BY_SPAN_KIND) is now enabled by default. This means spans with an eligible <span class="title-ref">span.kind</span> will have stats computed. If disabled, only top-level and measured spans will have stats computed. If you want to disable this feature, set <span class="title-ref">compute_stats_by_span_kind</span> to <span class="title-ref">false</span> in your Agent configuration.
Note: When using peer_tags_aggregation and compute_stats_by_span_kind, a high cardinality of peer tags or APM resources can contribute to higher CPU and memory consumption. If enabling both causes the Agent to consume too many resources, try disabling <span class="title-ref">compute_stats_by_span_kind</span> first.
It is recommended that you update your tracing libraries according to the instructions here and set DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED (or dd.trace.remove.integration-service-names.enabled) to true.
Upgraded JMXFetch to 0.49.5 which adds support for UnloadedClassCount metric and IBM J9 gc metrics. See 0.49.5 for more details.
Inferred Service dependencies are now Generally Available (exiting Beta) and enabled by default. Inferred Services of all kinds now have trace metrics and are available in dependency maps. <span class="title-ref">apm_config.peer_tags_aggregation</span> and <span class="title-ref">apm_config.compute_stats_by_span_kind</span> both now default to <span class="title-ref">true</span> unless explicitly set to <span class="title-ref">false</span>.
Add <span class="title-ref">check_tag_cardinality</span> parameter config check.
By default <span class="title-ref">check_tag_cardinality</span> is not set which doesn't change the behavior of the checks. Once it is set in pod annotaions, it overrides the cardinality value provided in the base agent configuration. Example of usage:
ad.datadoghq.com/redis.checks: |
{
"redisdb": {
"check_tag_cardinality": "high",
"instances": [
{
"host": "%%host%%",
"port": "6379"
}
]
}
}
system-probe/ebpf_btf_loader.log.system-probe directory.flare subcommand includes a --provider-timeout option to set a timeout for each file collection (default is 10s), useful for unblocking slow flare creation.system-probe/system_probe_telemetry.log.Released on: 2024-12-16 Pinned to datadog-agent v7.60.0: CHANGELOG.
pod_collection_mode set to node_kubelet. Previously, the check could fail to start if there was a timeout while contacting the API server. This issue has now been resolved.Release on: 2024-12-02
Release on: 2024-11-07
otlp_config.debug.loglevel in favor of otlp_config.debug.verbosity:
loglevel: debug maps to verbosity: detailedloglevel: info maps to verbosity: normalloglevel: warn/error maps to verbosity: basicloglevel: disabled maps to verbosity: noneKeepJSONPath - option to control whether JSON paths following JSON operators in SQL statements should be obfuscated. This option is only valid when ObfuscationMode is obfuscate_and_normalize.Added additional Agent telemetry metrics for the log tailer code flow: logs.bytes_sent, logs.encoded_bytes_sent, and logs.bytes_missed
Datadog may collect environmental, performance, and feature usage information about the Datadog Agent. This may include diagnostic logs and crash dumps of the Datadog Agent with obfuscated stack traces to support and further improve the Datadog Agent.
More details could be found in the docs
APM: Updates peer tags for peer.db.system.
Agents are now built with Go 1.22.8.
While using the AWS Lambda Extension, when a Lambda Function is invoked by a [properly instrumented][1] Step Function, the Lambda Function will create its Trace and Parent IDs deterministically based on the Step Function's execution context. [1]: https://docs.datadoghq.com/serverless/step_functions/installation/?tab=custom "Install Serverless Monitoring for AWS Step Functions"
Updates default .NET library used for auto-instrumentation from v2 to v3
The system-probe selinux policy is now installed on Oracle Linux
Increases the default input channel, processing channel, and context store sizes for network traffic paths.
Adds support for file log collection from Podman rootless containers when logs_config.use_podman_logs is set to true and podman_db_path is configured.
Allow Python integrations to emit Agent telemetry data.
Release on: 2024-11-04
Release on: 2024-10-24
Release on: 2024-10-21
active_session_history configuration parameter to optionally ingest Oracle active session history samples instead of query sampling.logs_config.tag_truncated_logs. When enabled, file logs will come with a tag truncated:true if they were truncated by the Agent.1.22.7.oracle_client_lib_dir config parameter.plan_hash_value is 0agent jmx [command] subcommands for container environments with annotations-based configs.tcp_sendpage probe.datadog-security-agent Windows service display name from Datadog Security Service to Datadog Security Agent for consistency with other Agent services.Released on: 2024-10-21 Pinned to datadog-agent v7.58.0: CHANGELOG.
Release on: 2024-09-24
1.22.7.Release on: 2024-09-17
Released on: 2024-09-17 Pinned to datadog-agent v7.57.1: CHANGELOG.
Release on: 2024-09-09
--json option to output the results in JSON format.Restricted Pod Security Standard, which require all containers to explicitly set a securityContext, an option to configure a <span class="title-ref">securityContext</span> to be used for all initContainers created by the auto instrumentation has been added. | This can be done through the DD_ADMISSION_CONTROLLER_AUTO_INSTRUMENTATION_INIT_SECURITY_CONTEXT environment value, or admission_controller.auto_instrumentation.init_security_context configuration -in both cases a json string should be supplied.filtering_enabled is set to true in the kubernetes_apiserver integration configuration.1.22.6.ddagentuser that Windows does not recognize.PROJECTLOCATION is misconfigured to an existing directory.DD_ORCHESTRATOR_EXPLORER_CUSTOM_SENSITIVE_ANNOTATIONS_LABELS to remove sensitive annotations and labels. For example: DD_ORCHESTRATOR_EXPLORER_CUSTOM_SENSITIVE_ANNOTATIONS_LABELS="sensitive-key-1 sensitive-key-2". Keys should be separated by spaces. The agent removes any annotations and labels matching these keys.Released on: 2024-09-09 Pinned to datadog-agent v7.57.0: CHANGELOG.
LimitRange and StorageClass resources are now collected by the orchestrator check.Release on: 2024-09-02
Released on: 2024-09-02 Pinned to datadog-agent v7.56.2: CHANGELOG.
Release on: 2024-08-29
Released on: 2024-08-29 Pinned to datadog-agent v7.56.1: CHANGELOG.
Release on: 2024-08-16
datadog.yaml file. The -E flag can be used to specify additional configuration files to be loaded sequentially after the main datadog.yaml.DD_SERVERLESS_STREAM_LOGS is enabled, DD_EXTENSION prints collected logs like agent stream-logs.Add the <span class="title-ref">use_apiserver_cache</span> option to the <span class="title-ref">kubernetes_state_metrics</span> check to reduce the pressure on the underlying storage engine <span class="title-ref">etcd</span>. Requires Kubernetes 1.19+.
APM: Add obfuscation support for OpenSearch statements within span metadata. This feature works in the same way as the existing Elasticsearch one, and is enabled by default. It is configured by binding apm_config.obfuscation.opensearch.* parameters to new obfuscation environment variables. In particular, bind: apm_config.obfuscation.opensearch.enabled to DD_APM_OBFUSCATION_OPENSEARCH_ENABLED: It accepts a boolean value with default value true.
apm_config.obfuscation.opensearch.keep_values to DD_APM_OBFUSCATION_OPENSEARCH_KEEP_VALUES It accepts a list of strings of the form ["id1", "id2"].
apm_config.obfuscation.opensearch.obfuscate_sql_values to DD_APM_OBFUSCATION_OPENSEARCH_OBFUSCATE_SQL_VALUES It accepts a list of strings of the form ["key1", "key2"].
Agents are now built with Go 1.22.4.
Agents are now built with Go 1.22.5.
Bump dependency <span class="title-ref">msodbcsql18</span> to version 18.3.3.1.
Adds config kubernetes_events_source_detection.enabled which is false by default. When set to true, this sets the source of kubernetes events to specific integrations based on the name of the controller that emitted it. All kubernetes events will have the tag orchestrator:kubernetes. For controller names that do not match any of the known integrations, the source will still be set to kubernetes by default.
Introduces a bundle_unspecifed_events config to the docker integration. When bundle_unspecifed_events and unbundle_events are true, Docker events are unbundled according to collected_event_types and the remaining events are bundled after excluding the filtered_event_types and collected_event_types.
The Agent will now ignore empty configuration files in conf.d. Users can use this functionality to avoid creating broken integrations when deploying agents with provisioning systems that do not allow skipping files entirely.
Introduces an bundle_unspecified_events config to the kubernetes_apiserver integration. When bundle_unspecified_events and unbundle_events are true, Kubernetes events are unbundled according to collected_event_types and the remaining events are bundled.
Improve utility functions that start or stop a service to better manage the SERVICE_START_PENDING and SERVICE_STOP_PENDING states by waiting for the service transition to complete. This will improve handling of concurrent execution of <span class="title-ref">agent.exe start</span> and <span class="title-ref">agent.exe restart</span> commands for the Windows Agent.
Make the <span class="title-ref">oom_kill</span> check capture the OOM score and the OOM score adjustment of the process being killed.
Oracle integration will now auto-adjust the size of the SQL substring requested by the activity sampler to better support users with multi-byte character sets, for example, Korean, Arabic, etc. This should alleviate crashes caused by long queries using these characters.
The OTLP ingestion endpoint now supports the same settings and protocol as the OpenTelemetry Collector OTLP receiver v0.103.0.
APM: Probabilistic Sampler now only looks at the lower 64 bits of a trace ID by default to improve compatibility in distributed systems where some apps may truncate the trace ID. To maintain the previous behavior use the feature flag <span class="title-ref">probabilistic_sampler_full_trace_id</span>.
Adds the source of the payload for Processes-owned messages.
Add tags to <span class="title-ref">CollectorManifest</span>
Add image and imageID fields to pods ContainerStatuses.
The orchestrator check can now scrub sensitive data from probes in pods specifications.
The Agent now populates the <span class="title-ref">git.repository_url</span> and <span class="title-ref">git.commit.sha</span> tags from the values of the <span class="title-ref">DD_GIT_REPOSITORY_URL</span> and <span class="title-ref">DD_GIT_COMMIT_SHA</span> container environment variables.
Implement the kubernetes_persistent_volume_claims_as_tags configuration that allows users to disable PersistentVolumeClaim for Kubernetes resources.
Upgrade the NTP check client library 'beevik/ntp' from v0.3.0 to v1.3.4
Use cloud-provided hostname as default when running the Agent in AKS.
APM: Enabled zstd compression by default on trace payloads.
datadog-agent status collector section.diagnose command for logs endpoints and related warnings about unknown config keys.Released on: 2024-08-16 Pinned to datadog-agent v7.56.0: CHANGELOG.
Release on: 2024-08-01
1.21.12.Released on: 2024-08-01 Pinned to datadog-agent v7.55.3: CHANGELOG.
Release on: 2024-07-25
CVE-2024-6257.Released on: 2024-07-25 Pinned to datadog-agent v7.55.2: CHANGELOG.
Release on: 2024-07-12
Released on: 2024-07-12 Pinned to datadog-agent v7.55.1: CHANGELOG.
The datadog-agent service fails to start on systems using the SysVInit init system, with a datadog-agent: unrecognized service error. See https://github.com/DataDog/datadog-agent/issues/27523 for details.
Release on: 2024-07-11
djm_config.enabled to enable Data Jobs MonitoringDD_LOGS_CONFIG_LAMBDA_LOGS_BUFFERING_TIMEOUT_MS, DD_LOGS_CONFIG_LAMBDA_LOGS_BUFFERING_MAX_BYTES, and DD_LOGS_CONFIG_LAMBDA_LOGS_BUFFERING_MAX_ITEMS.1.21.11.-l flag to the datadog-agent status command. This flag allows users to print all available sections.datadog-agent status component [section] in favor of datadog-agent status [section].CREATE TABLE IF NOT EXISTS.datadog-agent status and datadog-agent flare commands could hang while trying to reach other agent processes.Released on: 2024-07-11 Pinned to datadog-agent v7.55.0: CHANGELOG.
reason:backofflimitexceeded,deadlineexceeded to the kubernetes_state.job.failed metric to help users understand why a job failed.Release on: 2024-06-18
Released on: 2024-06-18 Pinned to datadog-agent v7.54.1: CHANGELOG.