The combination of the if
conditional and the Pipeline Processor can result in a simple,
yet powerful means to process heterogeneous input. For example, you can define a single pipeline
that delegates to other pipelines based on some criteria.
PUT _ingest/pipeline/logs_pipeline { "description": "A pipeline of pipelines for log files", "version": 1, "processors": [ { "pipeline": { "if": "ctx.service?.name == 'apache_httpd'", "name": "httpd_pipeline" } }, { "pipeline": { "if": "ctx.service?.name == 'syslog'", "name": "syslog_pipeline" } }, { "fail": { "message": "This pipeline requires service.name to be either `syslog` or `apache_httpd`" } } ] }
The above example allows consumers to point to a single pipeline for all log based index requests. Based on the conditional, the correct pipeline will be called to process that type of data.
This pattern works well with a default pipeline defined in an index mapping template for all indexes that hold data that needs pre-index processing.