2017-09-12

Apache Tamaya: Use Cases and Requirements

Use Cases

Simple Access

Users want to be able to access configuration in a unified way both in SE and EE. EE may provide additional mechanism, such as injection, but the SE mechanisms should work as well, so any code written in SE is fully portable to EE as well. This can only be achieved by providing a static accessor, e.g.

Configuration config = Configuration.current();

The call above should work exactly the same in EE. To enable this the static call must be delegated in the internals of the singleton, depending on the runtime. In EE the executing component can behave contextually, or even be loaded within the CDI environment (at least for post loading, application runtime aspects, but not earlier).

Additionally in EE it should also be possible to inject Configuration, which gives you the same results as the call above:

@Inject
private Configuration cfg;

Simple Lookup of Properties

Users just want to create a configuration ad hoc, from given property files. The files could be locally in the file system, on the classpath.

Tamaya should provide a simple Java API for accessing key/value based configuration. Hereby users want to access properties as simple String values.

Hereby returning Java 8 Optional values must be considered as well, instead of returning null.

Value Placeholders

Users just want to to be able to add placeholders to the values of configuration (not the keys). The mechanisms for resolving the placeholders hereby should be not constraint to one single lanmguage like EL. Instead of different replacement strategies should be selectable, e.g. by prefixing an expression with the name of the resolver that should do the work (eg "blabla ${env:HOME} using Java version ${sys:java.version}.". This allows resolution mechanism to be isolated easily and also allows to use simpler mechanisms, if no more complex ones like EL are required. This is especially useful to deal with low resource environment like ME.

Type Safe Properties

Users just want to access properties not only as Strings, but let Tamaya do the conversion to the required or the configred target type. By defauklt all java.ui.lang wrapper and primitive types should be supported, but also other common types like date/time types, math numeric types and more.

It must be possible that users can register their own custom types.

Finally users also want to be able to dynamically provide or override type adaption (conversion), when reading a value, for a certain key/value pair.

Configuration Templates

Users want to be able to let Tamaya implement an interface template based on configuration. This use case is pretty similar to the injection use case. Basically the values are not injected, but cached within the template proxy returned, e.g. as DynamicValue. Similarly it could even be possible to define callback methods (default methods) or register listeners to DynamicValue instances returned.

Templates hereby can easily be managed, but provide a excellent mechanism to provide type-safe configuration to clients in a very transparent way. Details can be controlled by using the same annotations as for normal configuration injection.

Java 8 Functional Support

Users want to be able to benefit from the new programming styles introduced with Java 8. Configuration should provide extension points for different aspects, where additional code can be hooked in easily. In short: were possible functional interfaces should be modelled.

Examples:

  • code that converts a configuration to another kind of configuration: UnaryOperator<Configuration>

  • code that creates any kind of result based on a configuration: Function<Configuration,T>

  • Adapters for type conversion are defined as Function<String,T>

  • Key, value filters ccan be modelled as BiFunction<String,String,String>

  • etc.

Configuration Locations

Users want to be able to

  • read configuration from different locations.

  • By default classpath and file resources are supported. But similarly remote access using a JSON ReST call should be possible.

  • Tamaya should define a JSON and XML format for configuration.

  • Configuration locations should be scannable using ant-styled resource patterns, if possible.

  • Scanning and reading logic can be modularized by using a ConfigReader interface.

Configuration Formats

Users want to be able to use the format they prefer.

  • properties, xml-properties and ini-format should be supported by default

  • Other/custom formats should be easily addable by registering or providing the format on configuration evaluation (read).

  • When possible Tamaya should figure out which format to be used and how the InputStream should be correctly interpreted.

Multiple Configurations

When systems grow they must be modularized to keep control. Whereas that sounds not really fancy, it leads to additional aspects to be considered by a configuration system.

  • Different code modules, libraries, plugins or products want to have their "own" separated configuration.

  • Similar it should be possible to add fully specific additional configurations.

The default configuration hereby should always be present, whereas additional configurations are optional. Users want to be able to check the availability of such an additional configuration.

Of course, additional configuration must be identifiable. The best way to do is to be discussed, nevertheless the mechanism must not depend on Java EE and the identifying keys must be serializable easily. Basically simple names are sufficient and woukld provide exact this required functionality.

External Configuration

Users want to be able to replace, override, extend or adapt any parts or all of an existing configuration from external sources. This also must be the case for multi-context environments, where the context identifiers are the only way to link to the correct remote configuration.

Context Dependent Configuration

In multi tenancy setups or complex systems a hierarchical/graph model of contexts for configurations is required, or different runtime contexts are executed in parallel within the same VN. What sounds normal for EE also may be the case for pure SE environments:

  • Users want to be able to model different layers of runtime context

  • Users want to identify the current layer, so configuration used may be adapted.

Dynamic Provisioning (UC8)

In Cloud Computing, especially the PaaS and SaaS areas a typical use case would be that an application (or server) is deployed, configured and started dynamically. Typically things are controlled by some "active controller components", which are capable of

  • creating new nodes (using IaaS services)

  • deploying and starting the required runtime platform , e.g. as part of a PaaS solution.

  • deploying and starting the application modules.

All these steps require some kind of configuration. As of today required files are often created on the target node before the systems are started, using proprietary formats and mechanism. Similarly accessing the configuration in place may require examining the file system or using again proprietary management functions. Of course, a configuration solution should not try to solve that, but it can provide a significant bunch of functionality useful in such scenarios:

  • provide remote capabilities for configuration

  • allow configuration to be updated remotely.

  • allow client code to listen for configuration changes and react as needed.

Minimal Property Source SPI

Users expect that implementing an additional configuration property source is as easy as possible. So there should be an SPI defined that allows any kind of data source to be used for providing configuration data. The interface to be implemented is expected to be minimal to reduce the implementation burden. Default methods should be used where possible, so only a few methods are expected to be required to implement.

Scannable Properties

If possible configuration should be scannable, meaning it should be possible to evaluate the keys available. The corresponding capabilities should be accessible by a isScannable() method.

Combine Configurations

Users want to be able to combine different configurations to a new configuration instance. Hereby the resulting configuration can be

  • a union of both, ignoring duplicates (and optionally log them)

  • a union of both, duplicates are ignored

  • a union of both, conflicts are thrown as ConfigException

  • an intersection of both, containing only keys present and equal in both configurations

  • an arbitrary mapping or filter, modelled by an CombinationPolicy, which basically can be modelled as BiFunction<String, String, String>, hereby

    • a result of null will remove the key

    • any other result will use the value returned as final value of the combination.

MX/ReST Management

Users want to be have comprehensive management support, which should allow

  • to change configuration

  • to remove configuration

  • to view configuration and its provider details

Adaptable Service Context

Tamaya should support an adaptable ServiceContext to resolve any kind of implememntation services, both API services as core framework services. The ServiceContext should be dynamically replecable by configuring an alternate instance of using the Java *ServiceContet+.

This decouples component usage from its load and management part and als greatly simplifies integration with new/alternate runtime environments. The service context is exptected to provide

  • single singleton instances: these service can be cached.

  • access to multiple instances which implement some commons SPI interface.

  • as useful priorization of components is done by the model itself.

Configuration Injection

Users want to be able to polulate configured items by injecting configured values. Hereby

  • the lifecycle of the instances is not managed by Tamaya

  • all references to items configured are managed as weak references, to prevent memoryleaks.

  • Injection should if possible evaluate the properties by defaults. Even properties without any annotations are possible.

  • Users expect to exclude certain properties from calculation

  • Beside injection of properties it is also possible to call setter methods with one parameter similarly.

  • Basically injection is performed, when the instance is passed to the Tamaya configuration system. It should also be possible to inject/provide final values, especially Strings. Changes on configured values should be reflected in the current value. Setters methods similarly can be called again, with the new values, on changes.

  • Users expect to control dynamic values and recall of setter methods, basically the following strategies should be supported:

    • inject only once and ignore further changes.

    • reinject/reinitialize on each change

  • Dynamic Values can easily be modeled as ConfiguredValue instances, which should have the following functionality:

    • access the current value

    • access the new value

    • access the latest value access time in ms

    • access the latest value update time in ms

    • evaluate easily if the value has changed since the last access

    • accept the change

      • as a shortcut it should be possible to accept the change on access of the value implicitly, hereby always accessing the latest valid value.

    • ignore the change

    • register Consumer<DynamicValue> liasteners to listen on the changes (ans also removing them later again).

All observing functionality can be done completely asynchronously and in parallel.

Requirements

Core Configuration Requirements

General

Tamaya must provide a Java SE API for accessing key/value based configuration. Hereby

  • Configuration is modelled by an interface

  • Configuration is organized as key/value pairs, using a subset of functionality present on Map<String,String> as follows:

    • access a value by key (get)

    • check if a value is present (containsKey)

    • get a set of all defined keys (keySet)

    • a configuration must be convertible to a Map, by calling toMap()

    • a configuration must provide access to its meta information.

  • Configuration value access methods must never return null.

  • The API must support undefined values.

  • The API must support passing default values, to be returned if a value is undefined.

  • The API must allow to throw exceptions, when a value is undefined. Customized exceptions hereby should be supported.

  • Properties can be stored in the classpath, on a file or accessible by URL.

  • Properties can be stored minimally in properties, xml-properties or ini-format.

Minimalistic Property Source

For enabling easy integration of custom built configuration sources a minimalistic API/SPI must be defined, that

  • is modelled by an interface

  • is a minimal subset of Configuration necessary to implement a configuration.

  • must be convertible to a "Configuration+.

Extension Points

For supporting more complex scenarios, Configuration

  • must implement the composite pattern, meaning new Configuration instances can be created by combining existing configurations.

  • must be adaptable, by creating a new configuration by applying a UnaryOperator<COnfiguration> to it.

  • must be queryable, by passing a ConfigQuery to an Configuration instance.

Type Safety

Besides Strings Configuration should also support the following types:

  • Primitive types

  • Wrapper types

  • All other types (by using a PropertyAdapter

Hereby type conversion should be done as follows:

  1. Check if for the given target type an explicit adapter is registered, if so, use the registered adapter.

  2. If no adapter is present, check if the target type T has static methods called T of(String), T getInstance(String), T valueOf(String), T from(String). If so use this method to create the non value of T.

  3. Check if the target type has a constructor T(String). If so, try to instantiate an instance using the constructor.

  4. Give up, throw a IllegalArgument exception.

Configuration Fomats

By default Tamaya support the following configuration formats:

  • .properties

  • .xml properties

  • .ini files

It must be possible to add additional formats by registering them with the current ServiceContext.

Mutability

  • Configurations can be mutable, mutability can be accessed as a property.

  • Configuration can be changed by collecting the changes into a ConfigCHangeSet and apply this set to the given Configuration instance.

  • Besides the points above, Configuration is immutable.

Serializability and Immutability of Configuration

  • Configuration is modelled as a service. Therefore serialization may not work. This can be mitigated by adding a freeze feature, where the know key/value pairs are extracted into an immutable and serializable form.

Configuration Combination Requirements

At least the following composition policies must be supported:

  • override: subsequent entries override existing ones.

  • aggregate-exception: key/values were added, in case of conflicts a ConfigException must be thrown.

  • aggregate-ignore-duplicates: similar to union, whereas duplicates are ignored (leaving the initial value loaded).

  • aggregate-combine: conflicting entries were resolved by adding them both to the target configuration by redefining partial keys.

  • custom: any function determining the key/values to be kept must be possible

When combining configuration it must also be possible to override (file/classpath) configuration by

  • system properties.

  • command line arguments.

Configuration Injection

As metnioned configuration can be injected by passing a unconfigured instance of an annotated class to the Configuration.configure static method:

Configuring a POJO
MyPojo instance = new MyPojo();
Configuration.configure(instance);

Hereby * It must be possible to define default values to be used, if no valid value is present. * It must be possible to define dynamic expressions, at least for default values. * The values configured can be reinjected, if the underlying configuration changes. This should also be the case for final classes, such as Strings. * Reinjection should be controllable by an loading policy. * It must be possible to evaluate multiple keys, e.g. current keys, and as a backup deprecated keys from former application releases. * It must be possible to evaluate multiple configurations. * The type conversion of the properties injected must be configurable, by defining a PropertyAdapter. * The value evaluated for a property (before type conversion) must be adaptable as well. * It must be possible to observe configuration changes.

The following annotations must be present at least:

  • @ConfiguredProperty defining the key of the property to be evaluated. It takes an optional value, defining the property name. It must be possible to add multiple annotations of this kind to define an order of evaluation of possible keys.

  • @DefaultValue (optional) defines a default String value, to be used, when no other key is present.

  • @WithConfig (optional) defines the name of the configuration to be used. Similar to @ConfiguredProperty multiple configuration can be defined for lookup.

  • @WithConfigOperator allows to adapt the String value evaluated, before it is passed as input to injection or type conversion.

  • @WithPropertyAdapter allows to adapt the conversion to the required target type, hereby overriding any default conversion in place.

  • @WithLoadPolicy allows to define the policy for (re)injection of configured values.

  • @ObservesConfigChange allows to annotate methods that should be called on configuration changes.

  • *@DefaultAreas" allows to define a key prefix key to be used for the configured key, if no absolute key is defined.

Configuration Templates

For type safe configuration clients should be able to define an interface and let it implement by the configuration system based on Configuration available:

  • Clients define an interface and annotate it as required (similar to above)

  • The interface methods must not take any arguments

  • The configuration system can be called to return such an interface implementation.

  • The configuration system returns a proxy hereby providing type-safe access the values required.

  • Similar to configured types also templates support multiple values and custom adapters.

  • It is possible to listen on configuration changes for templates, so users of the templates may react on configuration changes.

The following snippet illustrates the requirements:

Type Safe Configuration Template Example
public interface MyConfig {

  @ConfiguredProperty("myCurrency")
  @DefaultValue("CHF")
  String getCurrency();

  @ConfiguredProperty("myCurrencyRate")
  Long getCurrencyRate();

  @ConfigChange
  default configChanged(ConfigChange event){
     ...
  }

}

Templates can be accessed by calling the Configuration.current(Class) method:

Accessing a type safe Configuration Template
MyConfig config = Configuration.current(MyConfig.class);

Server Configuration Requirements

  • Ensure Configuration can be transferred over the network easily.

  • Beside serializability text based formats for serialization in XML and JSON must be defined.

  • A management API must be defined, which allows to inspect the configuration in place, e.g. using JMX or REST services.

Java EE leads to the following requirements:

  • Configuration must be contextual, depending on the current runtime context (e.g. boot level, ear, war, …​).

  • Hereby contextual aspects can even exceed the levels described above, e.g. for SaaS scenarios.

  • Resources can be unloaded, e.g. wars, ears can be restarted.

  • The different contextual levels can also be used for overriding, e.g. application specific configuration may override ear or system configuration.

  • Configuration may be read from different sources (different classloaders, files, databases, remote locations).

  • Configuration may be read in different formats (deployment descriptors, ServiceLoader configuration, alt-DD feature, …​)

  • JSF also knows the concept of stages.

  • Many SPI’s of Java EE require the implementation of some well defined Java interface, so it would be useful if the configuration solution supports easy implementation of such instances.

  • In general it would be useful to model the Environment explicitly.

  • Configuration used as preferences is writable as well. This requires mutability to be modelled in way, without the need of synchronization.

  • JNDI can be used for configuration as well.

Configurations made in the tenant or user layer override the default app configuration etc., so

  • It must be possible to structure Configuration in layers that can override/extend each other.

  • The current environment must be capable of mapping tenant, user and other aspects, so a corresponding configuration (or layer) can be derived.

Extensions Requirements

It must be possible to easily add additional functionality by implementing external functional interfaces operating on Configuration.

  • UnaryOperator<Configuration> for converting into other version of Configuration.

  • ConfigQuery<T> extending Function<T, Configuration>.

Non Functional Requirements

THe following non-functional requirements must be met:

  • tbd