org.apache.kafka.streams

Class StreamsBuilder

java.lang.Object

org.apache.kafka.streams.StreamsBuilder

public class StreamsBuilder
extends Object

StreamsBuilder provide the high-level Kafka Streams DSL to specify a Kafka Streams topology.

See Also:

Topology, KStream, KTable, GlobalKTable

Constructor Summary

Constructors

Constructor and Description
StreamsBuilder()

Method Summary

Modifier and Type	Method and Description
StreamsBuilder	addGlobalStore(StateStoreSupplier<KeyValueStore> storeSupplier, String sourceName, Deserializer keyDeserializer, Deserializer valueDeserializer, String topic, String processorName, ProcessorSupplier stateUpdateSupplier) Adds a global StateStore to the topology.
StreamsBuilder	addGlobalStore(StateStoreSupplier<KeyValueStore> storeSupplier, String sourceName, TimestampExtractor timestampExtractor, Deserializer keyDeserializer, Deserializer valueDeserializer, String topic, String processorName, ProcessorSupplier stateUpdateSupplier) Adds a global StateStore to the topology.
StreamsBuilder	addStateStore(StateStoreSupplier supplier, String... processorNames) Adds a state store to the underlying Topology.
Topology	build() Returns the Topology that represents the specified processing logic.
<K,V> GlobalKTable<K,V>	globalTable(Serde<K> keySerde, Serde<V> valueSerde, String topic) Create a GlobalKTable for the specified topic.
<K,V> GlobalKTable<K,V>	globalTable(Serde<K> keySerde, Serde<V> valueSerde, String topic, StateStoreSupplier<KeyValueStore> storeSupplier) Create a GlobalKTable for the specified topic.
<K,V> GlobalKTable<K,V>	globalTable(Serde<K> keySerde, Serde<V> valueSerde, String topic, String queryableStoreName) Create a GlobalKTable for the specified topic.
<K,V> GlobalKTable<K,V>	globalTable(Serde<K> keySerde, Serde<V> valueSerde, TimestampExtractor timestampExtractor, String topic, String queryableStoreName) Create a GlobalKTable for the specified topic.
<K,V> GlobalKTable<K,V>	globalTable(String topic) Create a GlobalKTable for the specified topic.
<K,V> GlobalKTable<K,V>	globalTable(String topic, String queryableStoreName) Create a GlobalKTable for the specified topic.
<K,V> KStream<K,V>	merge(KStream<K,V>... streams) Create a new instance of KStream by merging the given KStreams.
<K,V> KStream<K,V>	stream(Pattern topicPattern) Create a KStream from the specified topic pattern.
<K,V> KStream<K,V>	stream(Serde<K> keySerde, Serde<V> valueSerde, Pattern topicPattern) Create a KStream from the specified topic pattern.
<K,V> KStream<K,V>	stream(Serde<K> keySerde, Serde<V> valueSerde, String... topics) Create a KStream from the specified topics.
<K,V> KStream<K,V>	stream(String... topics) Create a KStream from the specified topics.
<K,V> KStream<K,V>	stream(TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, Pattern topicPattern) Create a KStream from the specified topic pattern.
<K,V> KStream<K,V>	stream(TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, String... topics) Create a KStream from the specified topics.
<K,V> KStream<K,V>	stream(Topology.AutoOffsetReset offsetReset, Pattern topicPattern) Create a KStream from the specified topic pattern.
<K,V> KStream<K,V>	stream(Topology.AutoOffsetReset offsetReset, Serde<K> keySerde, Serde<V> valueSerde, Pattern topicPattern) Create a KStream from the specified topic pattern.
<K,V> KStream<K,V>	stream(Topology.AutoOffsetReset offsetReset, Serde<K> keySerde, Serde<V> valueSerde, String... topics) Create a KStream from the specified topics.
<K,V> KStream<K,V>	stream(Topology.AutoOffsetReset offsetReset, String... topics) Create a KStream from the specified topics.
<K,V> KStream<K,V>	stream(Topology.AutoOffsetReset offsetReset, TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, Pattern topicPattern) Create a KStream from the specified topic pattern.
<K,V> KStream<K,V>	stream(Topology.AutoOffsetReset offsetReset, TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, String... topics) Create a KStream from the specified topics.
<K,V> KTable<K,V>	table(Serde<K> keySerde, Serde<V> valueSerde, String topic) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Serde<K> keySerde, Serde<V> valueSerde, String topic, StateStoreSupplier<KeyValueStore> storeSupplier) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Serde<K> keySerde, Serde<V> valueSerde, String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(String topic) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(String topic, StateStoreSupplier<KeyValueStore> storeSupplier) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(TimestampExtractor timestampExtractor, String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, Serde<K> keySerde, Serde<V> valueSerde, String topic) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, Serde<K> keySerde, Serde<V> valueSerde, String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, String topic) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, String topic, StateStoreSupplier<KeyValueStore> storeSupplier) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, String topic, StateStoreSupplier<KeyValueStore> storeSupplier) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, TimestampExtractor timestampExtractor, Serde<K> keySerde, Serde<V> valueSerde, String topic, String queryableStoreName) Create a KTable for the specified topic.
<K,V> KTable<K,V>	table(Topology.AutoOffsetReset offsetReset, TimestampExtractor timestampExtractor, String topic, String queryableStoreName) Create a KTable for the specified topic.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamsBuilder

public StreamsBuilder()

Method Detail

stream

public <K,V> KStream<K,V> stream(String... topics)

Create a KStream from the specified topics. The default "auto.offset.reset" strategy, default TimestampExtractor, and default key and value deserializers as specified in the config are used.

If multiple topics are specified there is no ordering guarantee for records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

topics - the topic names; must contain at least one topic name

Returns:

a KStream for the specified topics

stream

public <K,V> KStream<K,V> stream(Topology.AutoOffsetReset offsetReset,
                                 String... topics)

Create a KStream from the specified topics. The default TimestampExtractor and default key and value deserializers as specified in the config are used.

If multiple topics are specified there is no ordering guarantee for records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topics if no valid committed offsets are available

topics - the topic names; must contain at least one topic name

Returns:

a KStream for the specified topics

stream

public <K,V> KStream<K,V> stream(Pattern topicPattern)

Create a KStream from the specified topic pattern. The default "auto.offset.reset" strategy, default TimestampExtractor, and default key and value deserializers as specified in the config are used.

If multiple topics are matched by the specified pattern, the created KStream will read data from all of them and there is no ordering guarantee between records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

topicPattern - the pattern to match for topic names

Returns:

a KStream for topics matching the regex pattern.

stream

public <K,V> KStream<K,V> stream(Topology.AutoOffsetReset offsetReset,
                                 Pattern topicPattern)

Create a KStream from the specified topic pattern. The default TimestampExtractor and default key and value deserializers as specified in the config are used.

If multiple topics are matched by the specified pattern, the created KStream will read data from all of them and there is no ordering guarantee between records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the matched topics if no valid committed offsets are available

topicPattern - the pattern to match for topic names

Returns:

a KStream for topics matching the regex pattern.

stream

public <K,V> KStream<K,V> stream(Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 String... topics)

Create a KStream from the specified topics. The default "auto.offset.reset" strategy and default TimestampExtractor as specified in the config are used.

If multiple topics are specified there is no ordering guarantee for records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topics - the topic names; must contain at least one topic name

Returns:

a KStream for the specified topics

stream

public <K,V> KStream<K,V> stream(Topology.AutoOffsetReset offsetReset,
                                 Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 String... topics)

Create a KStream from the specified topics. The default TimestampExtractor as specified in the config is used.

If multiple topics are specified there is no ordering guarantee for records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topics if no valid committed offsets are available

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topics - the topic names; must contain at least one topic name

Returns:

a KStream for the specified topics

stream

public <K,V> KStream<K,V> stream(TimestampExtractor timestampExtractor,
                                 Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 String... topics)

Create a KStream from the specified topics. The default "auto.offset.reset" strategy as specified in the config is used.

If multiple topics are specified there is no ordering guarantee for records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

timestampExtractor - the stateless timestamp extractor used for this source KStream, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topics - the topic names; must contain at least one topic name

Returns:

a KStream for the specified topics

stream

public <K,V> KStream<K,V> stream(Topology.AutoOffsetReset offsetReset,
                                 TimestampExtractor timestampExtractor,
                                 Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 String... topics)

Create a KStream from the specified topics.

If multiple topics are specified there is no ordering guarantee for records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topics if no valid committed offsets are available

timestampExtractor - the stateless timestamp extractor used for this source KStream, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topics - the topic names; must contain at least one topic name

Returns:

a KStream for the specified topics

stream

public <K,V> KStream<K,V> stream(Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 Pattern topicPattern)

Create a KStream from the specified topic pattern. The default "auto.offset.reset" strategy and default TimestampExtractor as specified in the config are used.

If multiple topics are matched by the specified pattern, the created KStream will read data from all of them and there is no ordering guarantee between records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topicPattern - the pattern to match for topic names

Returns:

a KStream for topics matching the regex pattern.

stream

public <K,V> KStream<K,V> stream(Topology.AutoOffsetReset offsetReset,
                                 Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 Pattern topicPattern)

Create a KStream from the specified topic pattern. The default TimestampExtractor as specified in the config is used.

If multiple topics are matched by the specified pattern, the created KStream will read data from all of them and there is no ordering guarantee between records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the matched topics if no valid committed offsets are available

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topicPattern - the pattern to match for topic names

Returns:

a KStream for topics matching the regex pattern.

stream

public <K,V> KStream<K,V> stream(TimestampExtractor timestampExtractor,
                                 Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 Pattern topicPattern)

Create a KStream from the specified topic pattern. The default "auto.offset.reset" strategy as specified in the config is used.

If multiple topics are matched by the specified pattern, the created KStream will read data from all of them and there is no ordering guarantee between records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

timestampExtractor - the stateless timestamp extractor used for this source KStream, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topicPattern - the pattern to match for topic names

Returns:

a KStream for topics matching the regex pattern.

stream

public <K,V> KStream<K,V> stream(Topology.AutoOffsetReset offsetReset,
                                 TimestampExtractor timestampExtractor,
                                 Serde<K> keySerde,
                                 Serde<V> valueSerde,
                                 Pattern topicPattern)

Create a KStream from the specified topic pattern.

If multiple topics are matched by the specified pattern, the created KStream will read data from all of them and there is no ordering guarantee between records from different topics.

Note that the specified input topics must be partitioned by key. If this is not the case it is the user's responsibility to repartition the date before any key based operation (like aggregation or join) is applied to the returned KStream.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the matched topics if no valid committed offsets are available

timestampExtractor - the stateless timestamp extractor used for this source KStream, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to read this source KStream, if not specified the default serde defined in the configs will be used

valueSerde - value serde used to read this source KStream, if not specified the default serde defined in the configs will be used

topicPattern - the pattern to match for topic names

Returns:

a KStream for topics matching the regex pattern.

table

public <K,V> KTable<K,V> table(String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy, default TimestampExtractor, and default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null this is the equivalent of table(String)

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(String topic,
                               StateStoreSupplier<KeyValueStore> storeSupplier)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy and default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

topic - the topic name; cannot be null

storeSupplier - user defined state store supplier; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(String topic)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy and default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with an internal store name. Note that that store name may not be queriable through Interactive Queries. No internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

Parameters:

topic - the topic name; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. The default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null this is the equivalent of table(Topology.AutoOffsetReset, String)

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               String topic,
                               StateStoreSupplier<KeyValueStore> storeSupplier)

Create a KTable for the specified topic. The default TimestampExtractor and default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

topic - the topic name; cannot be null

storeSupplier - user defined state store supplier; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               String topic)

Create a KTable for the specified topic. The default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with an internal store name. Note that that store name may not be queriable through Interactive Queries. No internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

topic - the topic name; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(TimestampExtractor timestampExtractor,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy and default key and value deserializers as specified in the config are used. Input KeyValue pairs with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given storeName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(storeName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

timestampExtractor - the stateless timestamp extractor used for this source KTable, if not specified the default extractor defined in the configs will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               TimestampExtractor timestampExtractor,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. The default key and value deserializers as specified in the config are used. Input KeyValue pairs with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given storeName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(storeName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

timestampExtractor - the stateless timestamp extractor used for this source KTable, if not specified the default extractor defined in the configs will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy and default TimestampExtractor as specified in the config are used. Input records with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic,
                               StateStoreSupplier<KeyValueStore> storeSupplier)

Create a KTable for the specified topic. The default TimestampExtractor as specified in the config is used. The default "auto.offset.reset" strategy as specified in the config is used. Input records with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

storeSupplier - user defined state store supplier; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy as specified in the config is used. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with an internal store name. Note that that store name may not be queriable through Interactive Queries. No internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(TimestampExtractor timestampExtractor,
                               Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy as specified in the config is used. Input KeyValue pairs with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given storeName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(storeName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

timestampExtractor - the stateless timestamp extractor used for this source KTable, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               TimestampExtractor timestampExtractor,
                               Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic,
                               String queryableStoreName)

Create a KTable for the specified topic. Input KeyValue pairs with null key will be dropped.

Note that the specified input topic must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given storeName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(storeName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

timestampExtractor - the stateless timestamp extractor used for this source KTable, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic)

Create a KTable for the specified topic. The default "auto.offset.reset" strategy as specified in the config is used. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with an internal store name. Note that that store name may not be queriable through Interactive Queries. No internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

Returns:

a KTable for the specified topic

table

public <K,V> KTable<K,V> table(Topology.AutoOffsetReset offsetReset,
                               TimestampExtractor timestampExtractor,
                               Serde<K> keySerde,
                               Serde<V> valueSerde,
                               String topic,
                               StateStoreSupplier<KeyValueStore> storeSupplier)

Create a KTable for the specified topic. Input records with null key will be dropped.

Note that the specified input topics must be partitioned by key. If this is not the case the returned KTable will be corrupted.

The resulting KTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)
 

For non-local keys, a custom RPC mechanism must be implemented using KafkaStreams.allMetadata() to query the value of the key on a parallel running instance of your Kafka Streams application.

Parameters:

offsetReset - the "auto.offset.reset" policy to use for the specified topic if no valid committed offsets are available

timestampExtractor - the stateless timestamp extractor used for this source KTable, if not specified the default extractor defined in the configs will be used

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

storeSupplier - user defined state store supplier; cannot be null

Returns:

a KTable for the specified topic

globalTable

public <K,V> GlobalKTable<K,V> globalTable(String topic,
                                           String queryableStoreName)

Create a GlobalKTable for the specified topic. The default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

The resulting GlobalKTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key);
 

Note that GlobalKTable always applies "auto.offset.reset" strategy "earliest" regardless of the specified value in StreamsConfig.

Parameters:

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a GlobalKTable for the specified topic

globalTable

public <K,V> GlobalKTable<K,V> globalTable(String topic)

Create a GlobalKTable for the specified topic. The default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

The resulting GlobalKTable will be materialized in a local KeyValueStore with an internal store name. Note that that store name may not be queriable through Interactive Queries. No internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

Note that GlobalKTable always applies "auto.offset.reset" strategy "earliest" regardless of the specified value in StreamsConfig.

Parameters:

topic - the topic name; cannot be null

Returns:

a GlobalKTable for the specified topic

globalTable

public <K,V> GlobalKTable<K,V> globalTable(Serde<K> keySerde,
                                           Serde<V> valueSerde,
                                           TimestampExtractor timestampExtractor,
                                           String topic,
                                           String queryableStoreName)

Create a GlobalKTable for the specified topic. The default TimestampExtractor and default key and value deserializers as specified in the config are used. Input KeyValue pairs with null key will be dropped.

The resulting GlobalKTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key);
 

Note that GlobalKTable always applies "auto.offset.reset" strategy "earliest" regardless of the specified value in StreamsConfig.

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

timestampExtractor - the stateless timestamp extractor used for this source KTable, if not specified the default extractor defined in the configs will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a GlobalKTable for the specified topic

globalTable

public <K,V> GlobalKTable<K,V> globalTable(Serde<K> keySerde,
                                           Serde<V> valueSerde,
                                           String topic,
                                           StateStoreSupplier<KeyValueStore> storeSupplier)

Create a GlobalKTable for the specified topic. The default TimestampExtractor as specified in the config is used. Input KeyValue pairs with null key will be dropped.

The resulting GlobalKTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key);
 

Note that GlobalKTable always applies "auto.offset.reset" strategy "earliest" regardless of the specified value in StreamsConfig.

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

storeSupplier - user defined state store supplier; Cannot be null

Returns:

a GlobalKTable for the specified topic

globalTable

public <K,V> GlobalKTable<K,V> globalTable(Serde<K> keySerde,
                                           Serde<V> valueSerde,
                                           String topic,
                                           String queryableStoreName)

Create a GlobalKTable for the specified topic. The default TimestampExtractor as specified in the config is used. Input KeyValue pairs with null key will be dropped.

The resulting GlobalKTable will be materialized in a local KeyValueStore with the given queryableStoreName. However, no internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

To query the local KeyValueStore it must be obtained via KafkaStreams#store(...):

 KafkaStreams streams = ...
 ReadOnlyKeyValueStore<String, Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore());
 String key = "some-key";
 Long valueForKey = localStore.get(key);
 

Note that GlobalKTable always applies "auto.offset.reset" strategy "earliest" regardless of the specified value in StreamsConfig.

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

queryableStoreName - the state store name; if null an internal store name will be automatically given

Returns:

a GlobalKTable for the specified topic

globalTable

public <K,V> GlobalKTable<K,V> globalTable(Serde<K> keySerde,
                                           Serde<V> valueSerde,
                                           String topic)

Create a GlobalKTable for the specified topic. The default key and value deserializers as specified in the config are used. Input records with null key will be dropped.

The resulting GlobalKTable will be materialized in a local KeyValueStore with an internal store name. Note that that store name may not be queriable through Interactive Queries. No internal changelog topic is created since the original input topic can be used for recovery (cf. methods of KGroupedStream and KGroupedTable that return a KTable).

Note that GlobalKTable always applies "auto.offset.reset" strategy "earliest" regardless of the specified value in StreamsConfig.

Parameters:

keySerde - key serde used to send key-value pairs, if not specified the default key serde defined in the configuration will be used

valueSerde - value serde used to send key-value pairs, if not specified the default value serde defined in the configuration will be used

topic - the topic name; cannot be null

Returns:

a GlobalKTable for the specified topic

addStateStore

public StreamsBuilder addStateStore(StateStoreSupplier supplier,
                                    String... processorNames)

Adds a state store to the underlying Topology.

Parameters:

supplier - the supplier used to obtain this state store StateStore instance

processorNames - the names of the processors that should be able to access the provided store

Returns:

itself

Throws:

TopologyException - if state store supplier is already added

addGlobalStore

public StreamsBuilder addGlobalStore(StateStoreSupplier<KeyValueStore> storeSupplier,
                                     String sourceName,
                                     Deserializer keyDeserializer,
                                     Deserializer valueDeserializer,
                                     String topic,
                                     String processorName,
                                     ProcessorSupplier stateUpdateSupplier)

Adds a global StateStore to the topology. The StateStore sources its data from all partitions of the provided input topic. There will be exactly one instance of this StateStore per Kafka Streams instance.

A SourceNode with the provided sourceName will be added to consume the data arriving from the partitions of the input topic.

The provided ProcessorSupplier will be used to create an ProcessorNode that will receive all records forwarded from the SourceNode. This ProcessorNode should be used to keep the StateStore up-to-date. The default TimestampExtractor as specified in the config is used.

Parameters:

storeSupplier - user defined state store supplier

sourceName - name of the SourceNode that will be automatically added

keyDeserializer - the Deserializer to deserialize keys with

valueDeserializer - the Deserializer to deserialize values with

topic - the topic to source the data from

processorName - the name of the ProcessorSupplier

stateUpdateSupplier - the instance of ProcessorSupplier

Returns:

itself

Throws:

TopologyException - if the processor of state is already registered

addGlobalStore

public StreamsBuilder addGlobalStore(StateStoreSupplier<KeyValueStore> storeSupplier,
                                     String sourceName,
                                     TimestampExtractor timestampExtractor,
                                     Deserializer keyDeserializer,
                                     Deserializer valueDeserializer,
                                     String topic,
                                     String processorName,
                                     ProcessorSupplier stateUpdateSupplier)

Adds a global StateStore to the topology. The StateStore sources its data from all partitions of the provided input topic. There will be exactly one instance of this StateStore per Kafka Streams instance.

A SourceNode with the provided sourceName will be added to consume the data arriving from the partitions of the input topic.

The provided ProcessorSupplier will be used to create an ProcessorNode that will receive all records forwarded from the SourceNode. This ProcessorNode should be used to keep the StateStore up-to-date.

Parameters:

storeSupplier - user defined state store supplier

sourceName - name of the SourceNode that will be automatically added

timestampExtractor - the stateless timestamp extractor used for this source, if not specified the default extractor defined in the configs will be used

keyDeserializer - the Deserializer to deserialize keys with

valueDeserializer - the Deserializer to deserialize values with

topic - the topic to source the data from

processorName - the name of the ProcessorSupplier

stateUpdateSupplier - the instance of ProcessorSupplier

Returns:

itself

Throws:

TopologyException - if the processor of state is already registered

merge

public <K,V> KStream<K,V> merge(KStream<K,V>... streams)

Create a new instance of KStream by merging the given KStreams.

There is no ordering guarantee for records from different KStreams.

Parameters:

streams - the KStreams to be merged

Returns:

a KStream containing all records of the given streams

build

public Topology build()

Returns the Topology that represents the specified processing logic.

Returns:

the Topology that represents the specified processing logic