A datafeed resource has the following properties:
aggregations
chunking_config
{"mode": "manual", "time_span": "3h"}
datafeed_id
frequency
150s
.
indices
["it_ops_metrics"]
job_id
query
{"match_all": {"boost": 1}}
.
query_delay
60s
and 120s
. This randomness improves the query
performance when there are multiple jobs running on the same node.
script_fields
scroll_size
size
parameter that is used in Elasticsearch searches.
The default value is 1000
.
delayed_data_check_config
{"enabled": true, "check_window": "1h"}
See
Delayed Data Check Configuration Objects.
Datafeeds might be required to search over long time periods, for several months or years. This search is split into time chunks in order to ensure the load on Elasticsearch is managed. Chunking configuration controls how the size of these time chunks are calculated and is an advanced configuration option.
A chunking configuration object has the following properties:
mode
There are three available modes:
auto
manual
time_span
.
off
time_span
manual
.
For example: 3h
.
The datafeed can optionally search over indices that have already been read in
an effort to find if any data has since been added to the index. If missing data
is found, it is a good indication that the query_delay
option is set too low and
the data is being indexed after the datafeed has passed that moment in time. See
Working with delayed data.
This check only runs on real-time datafeeds
The configuration object has the following properties:
enabled
true
check_window
null
which causes an appropriate check_window
to be calculated
when the real-time datafeed runs.
The default check_window
span calculation is the max between 2h
or 8 * bucket_span
.
The get datafeed statistics API provides information about the operational progress of a datafeed. All of these properties are informational; you cannot update their values:
assignment_explanation
datafeed_id
node
(object) The node upon which the datafeed is started. The datafeed and job will be on the same node.
id
name
0-o0tOo
.
ephemeral_id
transport_address
127.0.0.1:9300
.
attributes
{"ml.max_open_jobs": "10"}
.
state
(string) The status of the datafeed, which can be one of the following values:
started
stopped