Fluentd logging driver
Estimated reading time: 4 minutesThe fluentd
logging driver sends container logs to the
Fluentd collector as structured log data. Then, users
can use any of the various output plugins of
Fluentd to write these logs to various
destinations.
In addition to the log message itself, the fluentd
log
driver sends the following metadata in the structured log message:
Field | Description |
---|---|
container_id |
The full 64-character container ID. |
container_name |
The container name at the time it was started. If you use docker rename to rename a container, the new name is not reflected in the journal entries. |
source |
stdout or stderr |
log |
The container log |
The docker logs
command is not available for this logging driver.
Usage
Some options are supported by specifying --log-opt
as many times as needed:
fluentd-address
: specify a socket address to connect to the Fluentd daemon, exfluentdhost:24224
orunix:///path/to/fluentd.sock
tag
: specify a tag for fluentd message, which interprets some markup, ex{{.ID}}
,{{.FullID}}
or{{.Name}}
docker.{{.ID}}
To use the fluentd
driver as the default logging driver, set the log-driver
and log-opt
keys to appropriate values in the daemon.json
file, which is
located in /etc/docker/
on Linux hosts or
C:\ProgramData\docker\config\daemon.json
on Windows Server. For more about
+configuring Docker using daemon.json
, see
+daemon.json.
The following example sets the log driver to fluentd
and sets the
fluentd-address
option.
{
"log-driver": "fluentd",
"log-opts": {
"fluentd-address": "fluentdhost:24224"
}
}
Restart Docker for the changes to take effect.
Note:
log-opt
configuration options in thedaemon.json
configuration file must be provided as strings. Boolean and numeric values (such as the value forfluentd-async-connect
orfluentd-max-retries
) must therefore be enclosed in quotes ("
).
To set the logging driver for a specific container, pass the
--log-driver
option to docker run
:
docker run --log-driver=fluentd ...
Before using this logging driver, launch a Fluentd daemon. The logging driver
connects to this daemon through localhost:24224
by default. Use the
fluentd-address
option to connect to a different address.
docker run --log-driver=fluentd --log-opt fluentd-address=fluentdhost:24224
If container cannot connect to the Fluentd daemon, the container stops
immediately unless the fluentd-async-connect
option is used.
Options
Users can use the --log-opt NAME=VALUE
flag to specify additional Fluentd logging driver options.
fluentd-address
By default, the logging driver connects to localhost:24224
. Supply the
fluentd-address
option to connect to a different address. tcp
(default) and unix
sockets are supported.
docker run --log-driver=fluentd --log-opt fluentd-address=fluentdhost:24224
docker run --log-driver=fluentd --log-opt fluentd-address=tcp://fluentdhost:24224
docker run --log-driver=fluentd --log-opt fluentd-address=unix:///path/to/fluentd.sock
Two of the above specify the same address, because tcp
is default.
tag
By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the log tag option documentation for customizing the log tag format.
labels, env, and env-regex
The labels
and env
options each take a comma-separated list of keys. If
there is collision between label
and env
keys, the value of the env
takes
precedence. Both options add additional fields to the extra attributes of a
logging message.
The env-regex
option is similar to and compatible with env
. Its value is a
regular expression to match logging-related environment variables. It is used
for advanced log tag options.
fluentd-async-connect
Docker connects to Fluentd in the background. Messages are buffered until the
connection is established. Defaults to false
.
fluentd-buffer-limit
The amount of data to buffer before flushing to disk. Defaults to the amount of RAM available to the container.
fluentd-retry-wait
How long to wait between retries. Defaults to 1 second.
fluentd-max-retries
The maximum number of retries. Defaults to 10
.
fluentd-sub-second-precision
Generates event logs in nanosecond resolution. Defaults to false
.
Fluentd daemon management with Docker
About Fluentd
itself, see the project webpage
and its documents.
To use this logging driver, start the fluentd
daemon on a host. We recommend
that you use the Fluentd docker
image. This image is
especially useful if you want to aggregate multiple container logs on each
host then, later, transfer the logs to another Fluentd node to create an
aggregate store.
Test container loggers
-
Write a configuration file (
test.conf
) to dump input logs:<source> @type forward </source> <match *> @type stdout </match>
-
Launch Fluentd container with this configuration file:
$ docker run -it -p 24224:24224 -v /path/to/conf/test.conf:/fluentd/etc/test.conf -e FLUENTD_CONF=test.conf fluent/fluentd:latest
-
Start one or more containers with the
fluentd
logging driver:$ docker run --log-driver=fluentd your/application