Writing new data backends

Arguments

Return type

The function must either call the context object’s not_found method, or return a hash of lookup keys and their associated values. The hash can be empty.

function mymodule::hiera_backend(
  Hash                  $options,
  Puppet::LookupContext $context,
)

Ruby example signature:

dispatch :hiera_backend do
  param 'Hash', :options
  param 'Puppet::LookupContext', :context
end

The returned hash can include the lookup_options key to configure merge behavior for other keys. See Configuring merge behavior in Hiera data for more information. Values in the returned hash can include Hiera interpolation tokens like %{variable} or %{lookup('key')}; Hiera interpolates values as needed. This is a significant difference between data_hash and the other two backend types; lookup_key and data_dig functions have to explicitly handle interpolation.

You can view its source on Git at eyaml_lookup_key.rb.

Arguments

Return type

The function must either call the context object’s not_found method, or return a value for the requested key. It can return undef as a value.

function mymodule::hiera_backend(
  Variant[String, Numeric] $key,
  Hash                     $options,
  Puppet::LookupContext    $context,
)

dispatch :hiera_backend do
  param 'Variant[String, Numeric]', :key
  param 'Hash', :options
  param 'Puppet::LookupContext', :context
end

Arguments

Return type

function mymodule::hiera_backend(
  Array[Variant[String, Numeric]] $segments,
  Hash                            $options,
  Puppet::LookupContext           $context,
)

dispatch :hiera_backend do
  param 'Array[Variant[String, Numeric]]', :segments
  param 'Hash', :options
  param 'Puppet::LookupContext', :context
end

A data_dig function can return a hash for the the lookup_options key to configure merge behavior for other keys. See Configuring merge behavior in Hiera data for more info.

To support Hiera interpolation tokens like %{variable} or %{lookup('key')} in your data, call context.interpolate on your values before returning them.

In Ruby functions, the context object is a normal Ruby object of class Puppet::LookupContext, and you can call methods with standard Ruby syntax, for example context.not_found.

In Puppet language functions, the context object appears as the special data type Puppet::LookupContext, that has methods attached.You can call the context’s methods using Puppet’s chained function call syntax with the method name instead of a normal function call syntax, for example, $context.not_found. For methods that take a block, use Puppet’s lambda syntax (parameters outside block) instead of Ruby’s block syntax (parameters inside block).

not_found()

Tells Hiera to halt this lookup and move on to the next data source. Call this method when your function cannot find a matching key or a given lookup. This method returns no value.

For data_hash backends, return an empty hash. The empty hash results in not_found, and prevents further calls to the provider. Missing data sources are not an issue when using path, paths, glob, or globs, but are important for backends that locate their own data sources.

For lookup_key and data_dig backends, use not_found when a requested key is not present in the data source or the data source does not exist. Do not return undef or nil for missing keys, as these are legal values that can be set in data.

interpolate(value)

Returns the provided value, but with any Hiera interpolation tokens (%{variable} or %{lookup('key')}) replaced by their value. This lets you opt-in to allowing Hiera-style interpolation in your backend’s data sources. It works recursively on arrays and hashes. Hashes can interpolate into both keys and values.

In data_hash backends, support for interpolation is built in, and you do not need to call this method.

In lookup_key and data_dig backends, call this method if you want to support interpolation.

environment_name()

Returns the name of the environment, regardless of layer.

module_name()

Returns the name of the module whose hiera.yaml called the function. Returns undef (in Puppet) or nil (in Ruby) if the function was called by the global or environment layer.

cache(key, value)

Caches a value, in a per-data-source private cache. It also returns the cached value.

On future lookups in this data source, you can retrieve values by calling cached_value(key). Cached values are immutable, but you can replace the value for an existing key. Cache keys can be anything valid as a key for a Ruby hash, including nil.

For example, on its first invocation for a given YAML file, the built-in eyaml_lookup_key backend reads the whole file and caches it, and then decrypts only the specific value that was requested. On subsequent lookups into that file, it gets the encrypted value from the cache instead of reading the file from disk again. It also caches decrypted values so that it won’t have to decrypt again if the same key is looked up repeatedly.

The cache is useful for storing session keys or connection objects for backends that access a network service.

Each Puppet::LookupContext cache lasts for the duration of the current catalog compilation. A node can’t access values cached for a previous node.

Hiera creates a separate cache for each combination of inputs for a function call, including inputs like name that are configured in hiera.yaml but not passed to the function. Each hierarchy level has its own cache, and hierarchy levels that use multiple paths have a separate cache for each path.

If any inputs to a function change, for example, a path interpolates a local variable whose value changes between lookups, Hiera uses a fresh cache.

cache_all(hash)

Caches all the key-value pairs from a given hash. Returns undef (in Puppet) or nil (in Ruby).

cached_value(key)

Returns a previously cached value from the per-data-source private cache. Returns undef or nil if no value with this name has been cached.

cache_has_key(key)

Checks whether the cache has a value for a given key yet. Returns true or false.

cached_entries()

Returns everything in the per-data-source cache as an iterable object. The returned object is not a hash. If you want a hash, use Hash($context.all_cached()) in the Puppet language or Hash[context.all_cached()] in Ruby.

cached_file_data(path)

Puppet syntax:

cached_file_data(path) |content| { ... }

Ruby syntax:

cached_file_data(path) {|content| ...}

For best performance, use this method to read files in Hiera backends.

context.cached_file_data(path) do |content|
      begin
        JSON.parse(content)
      rescue JSON::ParserError => ex
        # Filename not included in message, so we add it here.
        raise Puppet::DataBinding::LookupError, "Unable to parse (#{path}): #{ex.message}"
      end
    end

On repeated access to a given file, Hiera checks whether the file has changed on disk. If it hasn’t, Hiera uses cached data instead of reading and parsing the file again.

This method does not use the same per-data-source caches as cache(key, value) and similar methods. It uses a separate cache that lasts across multiple catalog compilations, and is tied to Puppet Server’s environment cache.

Because the cache can outlive a given node’s catalog compilation, do not do any node-specific pre-processing (like calling context.interpolate) in this method’s block.

explain() { ‘message’ }

Puppet syntax:

explain() || { 'message' }

Ruby syntax:

explain() { 'message' }

In both Puppet and Ruby, the provided block must take zero arguments.

explain() { 'message' } adds a message, which appears in debug messages or when using puppet lookup --explain. The block provided to this function must return a string.

The explain method is useful for complex lookups where a function tries several different things before arriving at the value. The built-in backends do not use the explain method, and they still have relatively verbose explanations. This method is for when you need to provide even more detail.

Hiera never executes the explain block unless explain is enabled.