wmts

Milestone: 3

This is a community-contributed plugin! It does not ship with logstash by default, but it is easy to install! To use this, you must have installed the contrib plugins package.

This filter converts data from OGC WMTS (Web Map Tile Service) URLs to geospatial information, and expands the logstash event accordingly. See http://www.opengeospatial.org/standards/wmts for more information about WMTS.

Given a grid, WMTS urls contain all the necessary information to find out which coordinates a requested tile belongs to. Using a simple grok filter you can extract all the relevant information. This plugin then translates these information into coordinates in LV03 and WGS84.

Here is an example of such a request: http://wmts4.geo.admin.ch/1.0.0/ch.swisstopo.pixelkarte-farbe/default/20130213/21781/23/470/561.jpeg

The current filter can be configured as follows in the configuration file:

filter { # First, waiting for varnish log file formats (combined apache logs) grok { match => [ “message”, “%{COMBINEDAPACHELOG}” ] } # Then, parameters grok { [ “request”, “(?([0-9\.]{5}))\/(?([a-z0-9\.-]*))\/default\/(?([0-9]*))\/(?([a-z0-9]*))\/(?([0-9]*))\/(?([0-9]*))\/(?([0-9]*))\.(?([a-zA-Z]*))" ] } # actually passes the previously parsed message to the wmts plugin wmts { } }

By default, the filter is configured to parse requests made on WMTS servers configured with the Swisstopo WMTS grid, but this can be customized, by setting the following parameters:

  • x_origin: the abscissa origin of the grid
  • y_origin: the ordinate origin of the grid
  • tile_width: the width of the produced image tiles
  • tile_height: the height of the image tiles
  • resolutions: the array of resolutions for this wmts grid

Additionnally, the following parameters can be set:

  • prefix: the prefix used on the added variables, by default ‘wmts.’
  • output_epsg: the output projection, classical one by default (lat/lon / epsg:4326)
  • zoomlevel_field: the name of the field where the filter can find the previously extracted zoomlevel, defaults to ‘wmts.zoomlevel’
  • column_field: same for column, defaults to ‘wmts.col’
  • row_field: same, defaults to ‘wmts.row’
  • refsys_field: same, defaults to ‘wmts.reference-system’ Note: if the reference system is different from the output_epsg, a reprojection of the coordinates will take place.
  • epsg_mapping: sometimes, the reference-system can be given as a string (‘swissgrid’ for instance). This parameter allows to set a mapping between a regular name and the epsg number of a projection, e.g.: { ‘swissgrid’ => 21781 }

Synopsis

This is what it might look like in your config file:
filter {
  wmts {
    add_field => ... # hash (optional), default: {}
    add_tag => ... # array (optional), default: []
    column_field => ... # string (optional), default: "wmts.col"
    epsg_mapping => ... # hash (optional), default: {}
    output_epsg => ... # string (optional), default: "epsg:4326"
    prefix => ... # string (optional), default: "wmts."
    refsys_field => ... # string (optional), default: "wmts.reference-system"
    remove_field => ... # array (optional), default: []
    remove_tag => ... # array (optional), default: []
    resolutions => ... # array (optional), default: [4000, 3750, 3500, 3250, 3000, 2750, 2500, 2250, 2000, 1750, 1500, 1250, 1000, 750, 650, 500, 250, 100, 50, 20, 10, 5, 2.5, 2, 1.5, 1, 0.5, 0.25, 0.1]
    row_field => ... # string (optional), default: "wmts.row"
    tile_height => ... # number (optional), default: 256
    tile_width => ... # number (optional), default: 256
    x_origin => ... # number (optional), default: 420000
    y_origin => ... # number (optional), default: 350000
    zoomlevel_field => ... # string (optional), default: "wmts.zoomlevel"
  }
}

Details

add_field

  • Value type is hash
  • Default value is {}

If this filter is successful, add any arbitrary fields to this event. Field names can be dynamic and include parts of the event using the %{field} Example:

filter {
  wmts {
    add_field => { "foo_%{somefield}" => "Hello world, from %{host}" }
  }
}

# You can also add multiple fields at once:

filter {
  wmts {
    add_field => { 
      "foo_%{somefield}" => "Hello world, from %{host}"
      "new_field" => "new_static_value"
    }
  }
}

If the event has field “somefield” == “hello” this filter, on success, would add field “foo_hello” if it is present, with the value above and the %{host} piece replaced with that value from the event. The second example would also add a hardcoded field.

add_tag

  • Value type is array
  • Default value is []

If this filter is successful, add arbitrary tags to the event. Tags can be dynamic and include parts of the event using the %{field} syntax. Example:

filter {
  wmts {
    add_tag => [ "foo_%{somefield}" ]
  }
}

# You can also add multiple tags at once:
filter {
  wmts {
    add_tag => [ "foo_%{somefield}", "taggedy_tag"]
  }
}

If the event has field “somefield” == “hello” this filter, on success, would add a tag “foo_hello” (and the second example would of course add a “taggedy_tag” tag).

column_field

  • Value type is string
  • Default value is "wmts.col"

configures the name of the field for the column

epsg_mapping

  • Value type is hash
  • Default value is {}

configures a mapping between named projections and their actual EPSG code. Some production WMTS use a regular name instead of a numerical value for the projection code. This parameter allows to define a custom mapping

exclude_tags DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value type is array
  • Default value is []

Only handle events without all/any (controlled by exclude_any config option) of these tags. Optional.

output_epsg

  • Value type is string
  • Default value is "epsg:4326"

configures the output projection

prefix

  • Value type is string
  • Default value is "wmts."

configures the prefix

refsys_field

  • Value type is string
  • Default value is "wmts.reference-system"

configures the name of the field for the reference system

remove_field

  • Value type is array
  • Default value is []

If this filter is successful, remove arbitrary fields from this event. Fields names can be dynamic and include parts of the event using the %{field} Example:

filter {
  wmts {
    remove_field => [ "foo_%{somefield}" ]
  }
}

# You can also remove multiple fields at once:

filter {
  wmts {
    remove_field => [ "foo_%{somefield}" "my_extraneous_field" ]
  }
}

If the event has field “somefield” == “hello” this filter, on success, would remove the field with name “foo_hello” if it is present. The second example would remove an additional, non-dynamic field.

remove_tag

  • Value type is array
  • Default value is []

If this filter is successful, remove arbitrary tags from the event. Tags can be dynamic and include parts of the event using the %{field} syntax. Example:

filter {
  wmts {
    remove_tag => [ "foo_%{somefield}" ]
  }
}

# You can also remove multiple tags at once:

filter {
  wmts {
    remove_tag => [ "foo_%{somefield}", "sad_unwanted_tag"]
  }
}

If the event has field “somefield” == “hello” this filter, on success, would remove the tag “foo_hello” if it is present. The second example would remove a sad, unwanted tag as well.

resolutions

  • Value type is array
  • Default value is [4000, 3750, 3500, 3250, 3000, 2750, 2500, 2250, 2000, 1750, 1500, 1250, 1000, 750, 650, 500, 250, 100, 50, 20, 10, 5, 2.5, 2, 1.5, 1, 0.5, 0.25, 0.1]

resolutions

row_field

  • Value type is string
  • Default value is "wmts.row"

configures the name of the field for the row

tags DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value type is array
  • Default value is []

Only handle events with all/any (controlled by include_any config option) of these tags. Optional.

tile_height

  • Value type is number
  • Default value is 256

tile_height

tile_width

  • Value type is number
  • Default value is 256

tile_width

type DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value type is string
  • Default value is ""

Note that all of the specified routing options (type,tags.exclude_tags,include_fields,exclude_fields) must be met in order for the event to be handled by the filter. The type to act on. If a type is given, then this filter will only act on messages with the same type. See any input plugin’s “type” attribute for more. Optional.

x_origin

  • Value type is number
  • Default value is 420000

WMTS grid configuration (by default, it is set to Swisstopo’s WMTS grid) x_origin

y_origin

  • Value type is number
  • Default value is 350000

y_origin

zoomlevel_field

  • Value type is string
  • Default value is "wmts.zoomlevel"

configures the name of the field for the WMTS zoomlevel


This is documentation from lib/logstash/filters/wmts.rb