gemfire

Milestone: 1

This is a community-contributed plugin! It does not ship with logstash by default, but it is easy to install! To use this, you must have installed the contrib plugins package.

Push events to a GemFire region.

GemFire is an object database.

To use this plugin you need to add gemfire.jar to your CLASSPATH. Using format=json requires jackson.jar too; use of continuous queries requires antlr.jar.

Note: this plugin has only been tested with GemFire 7.0.

Synopsis

This is what it might look like in your config file:
input {
  gemfire {
    add_field => ... # hash (optional), default: {}
    cache_name => ... # string (optional), default: "logstash"
    cache_xml_file => ... # string (optional), default: nil
    codec => ... # codec (optional), default: "plain"
    interest_regexp => ... # string (optional), default: ".*"
    query => ... # string (optional), default: nil
    region_name => ... # string (optional), default: "Logstash"
    serialization => ... # string (optional), default: nil
    tags => ... # array (optional)
    threads => ... # number (optional), default: 1
    type => ... # string (optional)
  }
}

Details

add_field

  • Value type is hash
  • Default value is {}

Add a field to an event

cache_name

  • Value type is string
  • Default value is "logstash"

Your client cache name

cache_xml_file

  • Value type is string
  • Default value is nil

The path to a GemFire client cache XML file.

Example:

 <client-cache>
   <pool name="client-pool" subscription-enabled="true" subscription-redundancy="1">
       <locator host="localhost" port="31331"/>
   </pool>
   <region name="Logstash">
       <region-attributes refid="CACHING_PROXY" pool-name="client-pool" >
       </region-attributes>
   </region>
 </client-cache>

charset DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value can be any of: "ASCII-8BIT", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "US-ASCII", "UTF-8", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "GB2312", "IBM437", "IBM737", "IBM775", "CP850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "IBM860", "IBM861", "IBM862", "IBM863", "IBM864", "IBM865", "IBM866", "IBM869", "Windows-1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "CP951", "stateless-ISO-2022-JP", "eucJP-ms", "CP51932", "GB12345", "ISO-2022-JP", "ISO-2022-JP-2", "CP50220", "CP50221", "Windows-1252", "Windows-1250", "Windows-1256", "Windows-1253", "Windows-1255", "Windows-1254", "TIS-620", "Windows-874", "Windows-1257", "Windows-31J", "MacJapanese", "UTF-7", "UTF8-MAC", "UTF-16", "UTF-32", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "BINARY", "CP437", "CP737", "CP775", "IBM850", "CP857", "CP860", "CP861", "CP862", "CP863", "CP864", "CP865", "CP866", "CP869", "CP1258", "Big5-HKSCS:2008", "eucJP", "euc-jp-ms", "eucKR", "eucTW", "EUC-CN", "eucCN", "CP936", "ISO2022-JP", "ISO2022-JP2", "ISO8859-1", "CP1252", "ISO8859-2", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "CP1256", "ISO8859-7", "CP1253", "ISO8859-8", "CP1255", "ISO8859-9", "CP1254", "ISO8859-10", "ISO8859-11", "CP874", "ISO8859-13", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "CP65000", "CP65001", "UTF-8-MAC", "UTF-8-HFS", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "external", "locale"
  • There is no default value for this setting.

The character encoding used in this input. Examples include “UTF-8” and “cp1252”

This setting is useful if your log files are in Latin-1 (aka cp1252) or in another character set other than UTF-8.

This only affects “plain” format logs since json is UTF-8 already.

codec

  • Value type is codec
  • Default value is "plain"

The codec used for input data. Input codecs are a convenient method for decoding your data before it enters the input, without needing a separate filter in your Logstash pipeline.

debug DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value type is boolean
  • Default value is false

format DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value can be any of: "plain", "json", "json_event", "msgpack_event"
  • There is no default value for this setting.

The format of input data (plain, json, json_event)

interest_regexp

  • Value type is string
  • Default value is ".*"

A regexp to use when registering interest for cache events. Ignored if a :query is specified.

message_format DEPRECATED

  • DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version.
  • Value type is string
  • There is no default value for this setting.

If format is “json”, an event sprintf string to build what the display @message should be given (defaults to the raw JSON). sprintf format strings look like %{fieldname}

If format is “json_event”, ALL fields except for @type are expected to be present. Not receiving all fields will cause unexpected results.

query

  • Value type is string
  • Default value is nil

A query to run as a GemFire “continuous query”; if specified it takes precedence over :interest_regexp which will be ignore.

Important: use of continuous queries requires subscriptions to be enabled on the client pool.

region_name

  • Value type is string
  • Default value is "Logstash"

The region name

serialization

  • Value type is string
  • Default value is nil

How the message is serialized in the cache. Can be one of “json” or “plain”; default is plain

tags

  • Value type is array
  • There is no default value for this setting.

Add any number of arbitrary tags to your event.

This can help with processing later.

threads

  • Value type is number
  • Default value is 1

Set this to the number of threads you want this input to spawn. This is the same as declaring the input multiple times

type

  • Value type is string
  • There is no default value for this setting.

Add a ‘type’ field to all events handled by this input.

Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can also use the type to search for it in the web interface.

If you try to set a type on an event that already has one (for example when you send an event from a shipper to an indexer) then a new input will not override the existing type. A type set at the shipper stays with that event for its life even when sent to another Logstash server.


This is documentation from lib/logstash/inputs/gemfire.rb