2017-11-25

Tamaya Formats (Extension Module)

Tamaya Formats is an extension module. Refer to the extensions documentation for further details.

What functionality this module provides ?

Tamaya Formats provides an abstraction for configuration formats provding the following benefits:

  • Parsing of resources in can be implemented separately from interpreting the different aspects/parts parsed. As an example a file format can define different sections. Depending on the company specific semantics of the sections a different set of PropertySource instances must be created.

  • Similarly the configuration abstraction can also be used as an interface for integrating Tamaya with alternate frameworks that provide logic for reading configuration files, such as Apache commons.configuration.

Compatibility

The module is based on Java 7, so it can be used with Java 7 and beyond.

Installation

To use the formats module you only must add the corresponding dependency to your module:

<dependency>
  <groupId>org.apache.tamaya.ext</groupId>
  <artifactId>tamaya-formats</artifactId>
  <version>{tamaya_version}</version>
</dependency>

Basic Concept

Formats should be reusable, meaning you should have to write a format parser only once and then be able to map the data read into whatever data structure (in our cases: property sources). So it is useful to separate concerns into

  • an arbitrary configuration format (textual or binary)

  • a parser (ConfigurationFormat) that transfers a given format into an intermediate representation (ConfigurationData).

  • an optional customization, implemented by a factory method pattern to adapt the mapping of ConfigurationData read to a collection of PropertySources (they can have different ordinal semantics).

ConfigurationData

Configuration formats can be very different. Some are simple key/value pairs, whereas other also consist of multiple sections (e.g. ini-files) or hierarchical data (e.g. yaml, xml). This is solved in Tamaya by mapping the configuration read into a normalized intermediary format called ConfigurationData:

ConfigurationData
public final class ConfigurationData {

    public ConfigurationFormat getFormat();
    public String getResource();

    public Set<String> getSectionNames();
    public Map<String,String> getSection(String name);

    public boolean hasDefaultProperties();
    public Map<String,String> getDefaultProperties();
    public Map<String,String> getCombinedProperties();

    public boolean isEmpty();
}

In detail the data read from a file is organized into sections as follows:

  • with getResource() and getFormat() the underlying resource and the format that read this data can be accessed.

  • properties can be owned by

    • named sections

    • an (unnamed) default section

  • each section section contains a map of properties. Hereby the same key can be part of the default section and multiple named sections, depending on the configuration format.

  • The method getSectionNames() returns a set of all section names.

  • With getSection(String name) a named section can be accessed.

  • With getDefaultSection() the 'default' section can be accessed. This is a convenience method.

  • With getCombinedProperties() a flattened entry map can be accessed built up (by default) out of

    • all entries from the default section, without any changes.

    • all entries from named sections, where the key for each entry is prefix with the section name and a '::' separator.

  • The configuration format used determines the mapping of configuration data read into this structure. The format implementation can as well provide alternate implementations of how the data read should be mapped into the combined properties map.

ConfigurationFormat

A ConfigurationFormat is basically an abstraction that reads a configuration resource (modelled by an InputStream) and creates a corresponding ConfigurationData instance.

public interface ConfigurationFormat {

    String getName();
    boolean accepts(URL url);
    ConfigurationData readConfiguration(String resource, InputStream inputStream);
}

Creating a default PropertySource for a ConfigurationFormat

The module defines a singleton ConfigurationFormats which provides an easy to use API for creating ConfigurationData and PropertySources using abstract ConfigurationFormat implementations:

public final class ConfigurationFormats {

    public static List<ConfigurationFormat> getFormats();
    public static List<ConfigurationFormat> getFormats(String... formatNames);
    public static List<ConfigurationFormat> getFormats(final URL url);

    public static ConfigurationData readConfigurationData(final URL url)
    throws IOException;
    public static ConfigurationData readConfigurationData(URL url, ConfigurationFormat... formats)
    throws IOException;
    public static ConfigurationData readConfigurationData(URL url, Collection<ConfigurationFormat> formats)
    throws IOException;
    public static Collection<ConfigurationData> readConfigurationData(Collection<URL> urls, ConfigurationFormat... formats);
    public static Collection<ConfigurationData> readConfigurationData(Collection<URL> urls, Collection<ConfigurationFormat> formats);
    public static ConfigurationData readConfigurationData(String resource, InputStream inputStream,
                                                          ConfigurationFormat... formats)
    throws IOException;
    public static ConfigurationData readConfigurationData(String resource, InputStream inputStream,
                                                          Collection<ConfigurationFormat> formats)
    throws IOException;

    public static PropertySource createPropertySource(URL url, ConfigurationFormat... formats)
    throws IOException;
    public static PropertySource createPropertySource(URL url, Collection<ConfigurationFormat> formats)
    throws IOException;
    public static PropertySource createPropertySource(String resource, InputStream inputStream,
                                                      ConfigurationFormat... formats);
    public static PropertySource createPropertySource(String resource, InputStream inputStream,
                                                       Collection<ConfigurationFormat> formats);
}
  • getFormats() returns all registered formats.

  • getFormats(String...) allows to access all formats with a given name.

  • getFormats(URL url) allows to access all formats that declare that can optionally read an input from a given URL.

  • readConfigurationData(...) reads data from an input and creates a corresponding ConfigurationData, either trying all known formats that declare its compatibility with the given input or the formats passed explicitly.

  • createPropertySource(...) allows to create a PropertySource reading a given input and the formats to be used or known. Hereby a default property mapping is applied.

So creating a PropertySource from a resource is basically a one liner:

URL url = ...;
PropertySource propertySource = ConfigurationFormats.createPropertySource(url);

// constraining the formats to be used (assumption: json and yaml extensions are loaded)
PropertySource propertySource = ConfigurationFormats.createPropertySource(
                                    url,
                                    ConfigurationFormats.getFormats("json", "yaml"));

Customize how ConfigurationData maps to PropertySource

For for the conversion of ConfigurationData into a PropertySource different approaches can be useful:

  1. The ConfigurationFormat that reads the data can provides all properties read either as sectioned properties or/and as default properties. The most simple cases is, where all properties have been added as 'default' properties. In this case the default properties can be used as the property sources properties without any change.

  2. If the format did also add section based properties, the combined properties returned can be used, hereby replacing the '::' separator with a '.' separator.

  3. In all other cases a custom mapping is useful, which can be acomplished by using the MappedConfigurationDataPropertySource and overriding the Map<String,String> populateData(ConfigurationData data) method.

In most cases the usage of a MappedConfigurationDataPropertySource, is a good choice to start. This class provides a convenient default mapping and also allows to customized the mapping easily:

ConfigurationData data = ...;
MappedConfigurationDataPropertySource ps =
  new MappedConfigurationDataPropertySource(data){
    protected Map<String, String> populateData(ConfigurationData data) {
      ...
    }
  };

Nevertheless, depending on the context, where a configuration source was read (classloader, time, source etc.) the resulting properties can have different semnatics, especially different priorities. Also section names may be mapped into different ordinals instead of using them as key prefixes (e.g. imagine configuration formats with a 'default', 'main', and 'overrides' sections). For such more complex or custom cases no simple mapping can be defined. Consequently the functionality mapping the normalized ConfigurationData read to the appropriate collection of PropertySource instances must be implemented.

For this scenario the BaseFormatPropertySourceProvider can be used, defining the following mapping function that mus be implemented:

/**
 * Method to create a {@link org.apache.tamaya.spi.PropertySource} based on the given entries read.
 *
 * @param data the configuration data, not null.
 * @return the {@link org.apache.tamaya.spi.PropertySource} instance ready to be registered.
 */
protected abstract Collection<PropertySource> getPropertySources(ConfigurationData data);

When using Java 8 these mappings can be asily passed as parameters to the createPropertySource methods.

Predefined formats

The formats module ships with 3 predefined formats:

  • .ini files, commonly known from Microsoft based systems, registered as ini.

  • .properties files, as defined by java.util.Properties, registered as properties.

  • .xml properties files, as defined by java.util.Properties, registered as xml-properties.

ini Configuration File Mapping

This module implements the ini file format with the class org.apache.tamaya.format.formats.IniConfigurationFormat.

The default mapping is bext illustrated by a small example, so consider the following .ini file:

a=valA
a.b=valB

[section1]
aa=sectionValA
aa.b.c=SectionValC

[section2]
a=val2Section2

This file content by default is mapped to the following Tamaya properties:

a=valA
a.b=valB
section1::valA=sectionValA
section1::a.b.c=SectionValC
section2::a=val2Section2

Summarizing

  • entries without a section are mapped to the default section.

  • entries with a section are mapped to a corresponding section, hereby everything between the brackets is used as section name (trimmed).

  • section names are separated using a double colon (::).

ConfigurationData allows to access all the different parts:

  • the default properties (a, a.b)

  • the section section1, with properties aa, aa.b.c

  • the section section2, with properties a

XML Property and ordinary Property Files

This module also ships with ConfigurationFormat implementations that reuse the parsing functionality provided with java.util.Properties:

  • org.apache.tamaya.format.formats.PropertiesFormat uses Properties.read(InputStream).

  • org.apache.tamaya.format.formats.PropertiesXmlFormat uses Properties.readFromXml(InputStream).