2017-09-12

Apache Tamaya: API

Though Tamaya is a very powerful and flexible solution there are basically only a few simple core concepts required. Everything else uses or extends these basic mechanisms. As a starting point we recommend you read the corresponding High Level Design Documentation

The Tamaya API

The API provides the artifacts as described in the High Level Design Documentation, which are:

  • The package org.apache.tamaya defines a simple but complete SE API for accessing key/value based Configuration:

    • Configuration hereby models configuration, the main interface of Tamaya. Configuration provides

      • access to literal key/value pairs.

      • functional extension points (with, query) using a unary ConfigOperator or a function ConfigurationQuery<T>.

    • ConfigurationProvider provides with getConfiguration() the static entry point for accessing configuration.

    • ConfigException defines a runtime exception for usage by the configuration system.

    • TypeLiteral provides a possibility to type safely define the target type to be returned in case non-String types are accessed.

    • PropertyConverter, which defines conversion of configuration values (String) into any required target type.

  • The package org.apache.tamaya.spi provides interfaces used for extending and/or adapting Tamaya’s core functionality, as well as artifacts for creating Configuration instances programmatically:

    • PropertySource: is the the interface to be implemented for adding configuration entries. A PropertySource hereby

      • is minimalistic and can be implemented in any way. E.g. there is no distiction that the configuration data provided is managed locally, remotedely. There is even no requirement that the configuration data is always fully available. Summarizing a PropertySource

      • provides property access for single key/value pairs in raw format (meaning no postprocessing is applied yet). A single property hereby is modelled as PropertyValue, which also includes its source and additional arbitrary metadata entries.

      • can optionally provide access to a Map<String,PropertyValue>, providing all its properties at once.

      • defines the default ordinal to be used for establishing the order of significance among all auto-discovered property sources.

    • PropertySourceProvider: allows to automatically register multiple property sources, e.g. all config files found in a file system folder..

    • ConfigurationProviderSpi defines the interface to be implemented by the delegating bean that is implementing the ConfigurationProvider singleton.

    • PropertyFilter allows filtering of property values prior getting returned to the caller. Filters by default are registered as global filters, filtering raw values. The final String value of a configuration entry is the final value after all registered filters have been applied.

    • A PropertyValueCombinationPolicy optionally can be registered to change the logic how key/value pairs from subsequent property sources in the property source chain are combined to calculate the final raw value passed over to the filters registered.

    • A ConfigurationContext is the container of all inner components (PropertySource, PropertyFilter, PropertyValueCombinationPolicy, PropertyConverter) required to implement a Configuration. Also the ordering of the property sources, filters and converters is defined by the context. A ConfigurationContext is automatically created on startup collecting and adding all registered artifacts. Based on this ConfigurationContext the default Configuration is created, which can be accessed from ConfigurationProvider.getConfiguration. Summarizing a ConfigurationContext contains the ordered property sources, property filters, converters and combination policy used. Once a ConfigurationContext is instanciated a corresponding Configuration instance can be created easily by calling ConfigurationProvider.createConfiguration(context).

    • Finally Tamaya also supports programmatically creating Configuration instances and using them in your code as needed. This can be achieved most easily by building a custom ConfigurationContext using a ConfigurationContextBuilder. This builder can be obtained calling ConfigurationProvider.getConfigurationContextBuilder();.

    • Finally ServiceContext and ServiceContextManager provide an abstraction to the underlying runtime environment, allowing different component loading and lifecycle strategies to be used. This is very useful since component (service) loading in Java SE, Java EE, OSGI and other runtime environments may be differ significantly. In most cases even extension programmers will not have to deal with these two artifacts.

Key/Value Pairs

Basically configuration is a very generic concept. Therefore it should be modelled in a generic way. The most simple and most commonly used approach are simple literal key/value pairs. So the core building block of {name} are key/value pairs. You can think of a common .properties file, e.g.

A simple properties file
a.b.c=cVal
a.b.c.1=cVal1
a.b.c.2=cVal2
a=aVal
a.b=abVal
a.b2=abVal

Now you can use java.util.Properties to read this file and access the corresponding properties, e.g.

Properties props = new Properties();
props.readProperties(...);
String val = props.getProperty("a.b.c");
val = props.getProperty("a.b.c.1");
...

Why Using Strings Only

There are good reason to keep of non String-values as core storage representation of configuration. Mostly there are several huge advantages:

  • Strings are simple to understand

  • Strings are human readable and therefore easy to prove for correctness

  • Strings can easily be used within different language, different VMs, files or network communications.

  • Strings can easily be compared and manipulated

  • Strings can easily be searched, indexed and cached

  • It is very easy to provide Strings as configuration, which gives much flexibility for providing configuration in production as well in testing.

  • and more…​

On the other side there are also disadvantages:

  • Strings are inherently not type safe, they do not provide validation out of the box for special types, such as numbers, dates etc.

  • In many cases you want to access configuration in a typesafe way avoiding conversion to the target types explicitly throughout your code.

  • Strings are neither hierarchical nor multi-valued, so mapping hierarchical and collection structures requires some extra efforts.

Nevertheless most of these advantages can be mitigated easily, hereby still keeping all the benefits from above:

  • Adding type safe adapters on top of String allow to add any type easily, that can be directly mapped out of Strings. This includes all common base types such as numbers, dates, time, but also timezones, formatting patterns and more.

  • Also multi-valued, complex and collection types can be defined as a corresponding PropertyAdapter knows how to parse and create the target instance required.

  • String s also can be used as references pointing to other locations and formats, where configuration is accessible.

[[API Configuration]]

Configuration

Configuration is the main artifact provided by Tamaya. It allows reading of single property values or all known properties, but also supports type safe access:

Interface Configuration
public interface Configuration{
    String get(String key);
    String getOrDefault(String key, String value);
    <T> T get(String key, Class<T> type);
    <T> T getOrDefault(String key, Class<T> type, T defaultValue);
    <T> T get(String key, TypeLiteral<T> type);
    <T> T getOrDefault(String key, TypeLiteral<T> type, T defaultValue);
    Map<String,String> getProperties();

    // extension points
    Configuration with(ConfigOperator operator);
    <T> T query(ConfigQuery<T> query);

    ConfigurationContext getContext();
}

Hereby

  • <T> T get(String, Class<T>) provides type safe accessors for all basic wrapper types of the JDK.

  • with, query provide the extension points for adding additional functionality.

  • getProperties() provides access to all key/values, whereas entries from non scannable property sources may not be included.

  • getOrDefault allows to pass default values as needed, returned if the requested value evaluated to null.

  • getConfigurationContext() allows access to the underlying components of a Configuration instance.

The class TypeLiteral is basically similar to the same class provided with CDI:

public class TypeLiteral<T> implements Serializable {

    [...]

    protected TypeLiteral(Type type) {
        this.type = type;
    }

    protected TypeLiteral() { }

    public static <L> TypeLiteral<L> of(Type type){...}
    public static <L> TypeLiteral<L> of(Class<L> type){...}

    public final Type getType() {...}
    public final Class<T> getRawType() {...}

    public static Type getGenericInterfaceTypeParameter(Class<?> clazz, Class<?> interfaceType){...}
    public static Type getTypeParameter(Class<?> clazz, Class<?> interfaceType){...}

    [...]
}

Instances of Configuration can be accessed from the ConfigurationProvider singleton:

Accessing Configuration
Configuration config = ConfigurationProvider.getConfiguration();

Hereby the singleton is backed up by an instance of ConfigurationProviderSpi, which is managed by the ServiceContextManager (see later).

Property Type Conversion

As illustrated in the previous section, Configuration also allows access of typed values. Internally all properties are strictly modelled as Strings. As a consequence non String values must be derived by converting the String values into the required target type. This is achieved with the help of PropertyConverters:

public interface PropertyConverter<T>{
    T convert(String value, ConversionContext context);
}

The ConversionContext contains additional meta-information about the key accessed, including the key’a name and additional metadata. This can be very useful, e.g. when the implementation of a PropertyConverter requires additional metadata for determining the correct conversion to be applied.

PropertyConverter instances can be implemented and registered by default using the Java ServiceLoader. The ordering of the registered converters, by default, is based on the annotated @Priority values (priority 0 is assumed if the annotation is missing). The first non-null result of a converter is returned as the final configuration value.

Access to converters is provided by the current ConfigurationContext, which is accessible calling Configuration.getConfigurationContext().

Extension Points

We are well aware of the fact that this library will not be able to cover all kinds of use cases. Therefore we have added functional extension mechanisms to Configuration that were used in other areas of the Java eco-system (e.g. Java Time API and JSR 354) as well:

  • with(ConfigOperator operator) allows to pass arbitrary unary functions that take and return instances of Configuration. Operators can be used to cover use cases such as filtering, configuration views, security interception and more.

  • query(ConfigQuery query) allows to apply a function returning any kind of result based on a Configuration instance. Queries are used for accessing/deriving any kind of data based on of a Configuration instance, e.g. accessing a Set<String> of root keys present.

Both interfaces hereby are functional interfaces. Because of backward compatibility with Java 7 we did not use UnaryOperator and Function from the java.util.function package. Nevertheless usage is similar, so you can use Lambdas and method references in Java 8:

Applying a ConfigurationQuery using a method reference
ConfigSecurity securityContext = ConfigurationProvider.getConfiguration().query(ConfigSecurity::targetSecurityContext);
Note
ConfigSecurity is an arbitrary class only for demonstration purposes.

Operator calls basically look similar:

Applying a ConfigurationOperator using a lambda expression:
Configuration secured = ConfigurationProvider.getConfiguration()
                           .with((config) ->
                                 config.get("foo")!=null?;
                                 FooFilter.apply(config):
                                 config);

ConfigException

The class ConfigException models the base runtime exception used by the configuration system.

SPI

PropertyValue, PropertyValueBuilder

On the API properties are represented as Strings only, whereas in the SPI value are represented as ProeprtyValue, which contain

  • the property’s key (String)

  • the property’s value (String)

  • the property’s source (String, typically equals to the property source’s name)

  • any additional meta-data represented as Map<String,String>

This helps to kepp all value relevant data together in one place and also allows to choose any kind of representation for meta-data entries. The PropertyValue itself is a final and serializable data container, which also has a powerful builder API (e.g. for using within filters):

public final class PropertyValue implements Serializable{
    [...]

    public static PropertyValue of(String key, String value, String source);

    public String getKey();
    public String getSource();
    public String getValue();
    public Map<String, String> getMetaEntries();
    public String getMetaEntry(String key);
    public PropertyValueBuilder toBuilder();

    public static PropertyValueBuilder builder(String key, String source);
    public static PropertyValueBuilder builder(String key, String value, String source);

    /**
     * Maps a map of {@code Map<String,String>} to a {@code Map<String,PropertyValue>}.
     * @param config the String based map, not null.
     * @param source the source name, not null.
     * @return the corresponding value based map.
     */
    public static Map<String,PropertyValue> map(Map<String, String> config, String source);

    /**
     * Maps a map of {@code Map<String,String>} to a {@code Map<String,PropertyValue>}.
     * @param config the String based map, not null.
     * @param source the source name, not null.
     * @param metaData additional metadata, not null.
     * @return the corresponding value based map.
     */
    public static Map<String,PropertyValue> map(Map<String, String> config, String source,
                                                Map<String,String> metaData);
}

When writing your own datasource you can easily create your own PropertyValues:

PropertyValue val = PropertyValue.of("key","value","source");

If you want to add additional metadata in most cases you would use the builder API:

PropertyValue val = PropertyValue.builder("key","value","source")
                     .addMetaEntry("figured", "true")
                     .build();

PropertyValues are type safe value objects. To change a value you have to create a new instance using a builder:

PropertyValue val = PropertyValue.builder("key","value","source")
                     .addMetaEntry("figured", "true")
                     .build();
PropertyValue newVal = val.toBuilder().setValue("anotehrValue")
                     .addMetaEntry("remote", "true")
                     .removeMetaEntry("figured")
                     .build();

Interface PropertySource

We have seen that constraining configuration aspects to simple literal key/value pairs provides us with an easy to understand, generic, flexible, yet extensible mechanism. Looking at the Java language features a java.util.Map<String, String> and java.util.Properties basically model these aspects out of the box.

Though there are advantages in using these types as a model, there are some drawbacks. Notably implementation of these types is far not trivial and the collection API offers additional functionality not useful when aiming for modelling simple property sources.

To render an implementation of a custom PropertySource as convenient as possible only the following methods were identified to be necessary:

public interface PropertySource{
      int getOrdinal();
      String getName();
      PropertyValue get(String key);
      boolean isScannable();
      Map<String,PropertyValue> getProperties();
}

Hereby

  • get looks similar to the methods on Map. It may return null in case no such entry is available.

  • getProperties allows to extract all property data to a Map<String,PropertyValue>. Other methods like containsKey, keySet as well as streaming operations then can be applied on the returned Map instance.

  • But not in all scenarios a property source is able to provide all values at once (aka to be scannable), e.g. when looking up keys is very inefficient, it may not make sense to iterate over all keys to collect the corresponding properties. If a PropertySource is defined as non scannable accesses to getProperties() may not return all key/value pairs that would be available when accessed directly using the PropertyValue get(String) method. The fact if a PropertySource is scannable can be determined by calling isScannable().

  • int getOrdinal() defines the ordinal of the PropertySource. Property sources are managed in an ordered chain, where property sources with higher ordinals override ones with lower ordinals. If the ordinal of two property sources is the same, the natural ordering of the fully qualified class names of the property source implementations is used. The reason for not using @Priority annotations is that property sources can define dynamically their ordinals, e.g. based on a property contained with the configuration itself. Implementations of this API may provide additional functionality to adapt the default ordinal of auto-discovered property sources.

  • Finally getName() returns a (unique) name that identifies the PropertySource within its containing ConfigurationContext.

This interface can be implemented by any kind of logic. It could be a simple in memory map, a distributed configuration provided by a data grid, a database, the JNDI tree or other resources. Or it can be a combination of multiple property sources with additional combination/aggregation rules in place.

PropertySources to be picked up automatically and be added to the default Configuration, must be registered using the Java +ServiceLoader (or the mechanism provided by the current active ServiceContext, see later in this document for further details).

Interface PropertySourceProvider

Instances of this type can be used to register multiple instances of PropertySource.

// @FunctionalInterface in Java 8
public interface PropertySourceProvider{
    Collection<PropertySource> getPropertySources();
}

This allows to evaluate the property sources to be read/that are available dynamically. All property sources are read out and added to the current chain of PropertySource instances within the current ConfigurationContext, refer also to .

PropertySourceProviders are by default registered using the Java ServiceLoader or the mechanism provided by the current active ServiceContext.

Interface PropertyFilter

Also PropertyFilters can be added to a Configuration. They are evaluated each time before a configuration value is passed to the user. Filters can be used for multiple purposes, such as

  • resolving placeholders

  • masking sensitive entries, such as passwords

  • constraining visibility based on the current active user

  • …​

For PropertyFilters to be picked up automatically and added to the default Configuration must be,by default, registered using the Java ServiceLoader (or the mechanism provided by the current active ServiceContext). Similar to property sources they are managed in an ordered filter chain, based on the class level @Priority annotations (assuming 0 if none is present).

A PropertyFilter is defined as follows:

// Functional Interface
public interface PropertyFilter{
    PropertyValue filterProperty(PropertyValue value, FilterContext context);
}

Hereby:

  • returning null will remove the key from the final result.

  • non null values are used as the current value of the key. Nevertheless for resolving multi-step dependencies filter evaluation has to be continued as long as filters are still changing some of the values to be returned. To prevent possible endless loops after a defined number of loops evaluation is stopped.

  • FilterContext provides additional metdata, including the property accessed, which is useful in many use cases.

This method is called each time a single entry is accessed, and for each property in a full properties result.

Interface PropertyValueCombinationPolicy

This interface is purely optional and can be used to adapt the way how property key/value pairs are combined to build up the final configuration raw value to be passed over to the PropertyFilters. The default implementation is just overriding all values read before with the new value read. Nevertheless for collections and other use cases more intelligent logic is required.

// FunctionalInterface
public interface PropertyValueCombinationPolicy{

   PropertyValueCombinationPolicy DEFAULT_OVERRIDING_COLLECTOR =
     new PropertyValueCombinationPolicy(){
       @Override
       public PropertyValue collect(PropertyValue currentValue, String key,
                                         PropertySource propertySource) {
           PropertyValue value = propertySource.get(key);
           return value!=null?value:currentValue;
       }
   };

   PropertyValue collect(PropertyValue currentValue, String key,
                  PropertySource propertySource);

}

Looking at the collect method’s signature, returning a PropertyValue allows also to filter/combine/use meta entries.

The Configuration Context

A Configuration is created from a ConfigurationContext. Each Configuration instance provides it’s underlying context by Configuration.getContext():

Accessing the current ConfigurationContext
Configuration config = ...;
ConfigurationContext context = config.getContext();

The ConfigurationContext provides access to the internal artifacts that determine the Configuration and also defines the ordering of the property sources, filters and converters contained:

  • PropertySources registered (including the PropertySources provided from PropertySourceProvider instances).

  • PropertyFilters registered, which filter values before they are returned to the client

  • PropertyConverter instances that provide conversion functionality for converting String values to any other types.

  • the current PropertyValueCombinationPolicy that determines how property values from different PropertySources are combined to the final property value returned to the client.

Changing the current Configuration Context

A ConfigurationContext is not mutable once it is created. In many cases mutability is also not needed. Nevertheless there are use cases where the current ConfigurationContext (and consequently Configuration) must be adapted:

  • New configuration files where detected in a folder observed by Tamaya.

  • Remote configuration, e.g. stored in a database or alternate ways has been updated and the current system must be adapted to these changes.

  • The overall configuration context is manually setup by the application logic.

  • Within unit testing alternate configuration setup should be setup to meet the configuration requirements of the tests executed.

In such cases the ConfigurationContext may change, meaning it must be possible:

  • to add or remove PropertySource instances

  • to add or remove PropertyFilter instances

  • to add or remove PropertyConverter instances

  • to redefine the current PropertyValueCombinationPolicy instances.

This is nothing new, a ConfigurationContextBuilder already allows us to achive this. So we must only ensure that we can easily create a new ConfigurationContextBuilder using a given ConfigurationContext as input. This can be done by

  • calling ConfigurationContext.toBuilder(), hereby returning a builder instance preinitialized with the values from the current ConfigurationContext.

  • calling ConfigurationProvider.getConfigurationContextBuilder() and then applying the current ConfigurationContext by calling setContext(ConfigurationContext) on the ConfigurationContextBuilder instance.

Accessing a ConfigurationContextBuilder
ConfigurationContextBuilder preinitializedContextBuilder = ConfigurationProvider.getConfiguration().getContext().toBuilder();
ConfigurationContextBuilder emptyContextBuilder = ConfigurationProvider.getConfigurationContextBuilder();

Finally when we are finished a new ConfigurationContext can be created:

Creating and applying a new ConfigurationContext
ConfigurationContext context = ConfigurationProvider.getConfiguration().getContext()
                                    .toBuilder();
                                    .addPropertySources(new MyPropertySource())
                                    .addPropertyFilter(new MyFilter())
                                    .build();

But let’s also have a deeper look at what functionality a ConfigurationContextBuilder provides. Basically such a builder allows to add, remove or reorder property sources, converters and filters or changing any other aspect of a ConfigurationContext:

Chain manipulation using ConfigurationContextBuilder
PropertySource propertySource = builder.getPropertySource("sourceId");

// changing the priority of a property source. The ordinal value hereby is not considered.
// Instead the position of the property source within the chain is changed.
builder.decreasePriority(propertySource);

// Alternately a comparator expression can be passed to establish the defined ordering...
builder.sortPropertyFilters(MyFilterComparator::compare);

Finally if the new ConfigurationContext is ready a new Configuration can be created. Optionally the new Configuration can also be installed as the new default Configuration instace as illustrated below:

Creating and applying a new ConfigurationContext
ConfigurationContext context = builder.build();

// Creates a new matching Configuration instance
Configuration newConfig = ConfigurationProvider.createConfiguration(context);

// Apply the new context to replace the current configuration:
ConfigurationProvider.setConfiguration(newConfig);

Hereby ConfigurationProvider.setConfiguration(Configuration) can throw an UnsupportedOperationException. This can be checked by calling the method boolean ConfigurationProvider.isConfigurationSettable().

Implementing and Managing Configuration

One of the most important SPI in Tamaya is the ConfigurationProviderSpi interface, which is backing up the ConfigurationProvider singleton. Implementing this class allows

  • to fully determine the implementation class for Configuration

  • to manage the current Configuration in the scope and granularity required.

  • to provide access to the right Configuration based on the current runtime context.

  • Performing changes as set with the current ConfigurationContextBuilder.

Interface ConfigurationContextBuilder

Interface ConfigurationContextBuilder

Overview

The Tamaya builder module provides a generic (one time) builder for creating Configuration instances, e.g. as follows:

ConfigurationBuilder builder = new ConfigurationBuilder();
// do something
Configuration config = builder.build();

Basically the builder allows to create configuration instances completely independent of the current configuration setup. This gives you full control how and when Configuration is created.

Supported Functionality

The builder allows you to add PropertySource instances:

ConfigurationContextBuilder builder = ConfigurationProvider.getConfigurationContextBuilder();
builder.addPropertySources(sourceOne, sourceTwo, sourceThree
Configuration config = ConfigurationProvider.createConfiguration(builder.build());

Hereby the ordering of the propertysources is not changed, regardless of the ordinals provided by the property sources. This allows alternate ordering policies easily being implemented because creating a configuration based on a configuration context is already implemented and provided by the core API.

Similarly you can add PropertyFilters:

builder.addPropertyFilters(new MyConfigFilter());

…​or PropertySourceProvider instances:

builder.addPropertySourceProvider(new MyPropertySourceProvider());

The ServiceContext

The ServiceContext allows to define how components are loaded in Tamaya. It is the glue layer, which interacts with the underlying runtime system such as Java SE, Java EE, OSGI, VertX etc. The ServiceContext hereby defines access methods to obtain components, whereas itself it is available from the ServiceContextManager singleton:

Accessing the ServiceContext
ServiceContext serviceContext = ServiceContextManager.getServiceContext();

public interface ServiceContext{
    int ordinal();
    <T> T getService(Class<T> serviceType);
    <T> List<T> getServices(Class<T> serviceType);
}

With the ServiceContext a component can be accessed in two different ways:

  1. access as as a single property. Hereby the registered instances (if multiple) are sorted by priority and then finally the most significant instance is returned only.

  2. access all items given a type. This will return (by default) all instances loadedable from the current runtime context, ordered by priority (the most significant components added first).

Examples

Accessing Configuration

Configuration is obtained from the ConfigurationProvider singleton:

Accessing Configuration
Configuration config = ConfigurationProvider.getConfiguration();

Many users in a SE context will probably only work with Configuration, since it offers all functionality needed for basic configuration with a very lean memory and runtime footprint. In Java 7 access to the keys is very similar to Map<String,String>, whereas in Java 8 additionally usage of Optional is supported:

Configuration config = ConfigurationProvider.getConfiguration();
String myKey = config.get("myKey");                         // may return null
int myLimit = config.get("all.size.limit", int.class);

Environment and System Properties

By default environment and system properties are included into the Configuration. So we can access the current PROMPT environment variable as follows:

String prompt = ConfigurationProvider.getConfiguration().get("PROMPT");

Similary the system properties are directly applied to the Configuration. So if we pass the following system property to our JVM:

java ... -Duse.my.system.answer=yes

we can access it as follows:

boolean useMySystem = ConfigurationProvider.getConfiguration().get("use.my.system.answer", boolean.class);

Adding a Custom Configuration

Adding a classpath based configuration is simply as well: just implement an according PropertySource. With the tamaya-spi-support module you just have to perform a few steps:

  1. Define a PropertySource as follows:

  public class MyPropertySource extends PropertiesResourcePropertySource{

    public MyPropertySource(){
        super(ClassLoader.getSystemClassLoader().getResource("META-INF/cfg/myconfig.properties"), DEFAULT_ORDINAL);
    }
  }

Then register MyPropertySource using the ServiceLoader by adding the following file:

META-INF/services/org.apache.tamaya.spi.PropertySource

…​containing the following line:

com.mypackage.MyPropertySource

API Implementation

The API is implemented by the tamaya-core module. Refer to the Core documentation for further details.