2017-02-09

Tamaya Model Documentation and Validation (Extension Module)

Tamaya Model is an extension module. Refer to the extensions documentation for further details.

What functionality this module provides ?

The Tamaya Model module provides support for documenting configuration and validating configuration read and processed against this model. Documentation and config models can be provided in different ways:

  • as separate meta-model documents

  • by providers that check classes/packages for configuration annotations (planned)

Compatibility

The module is based on Java 7, so it will not run on Java 7 and beyond.

Installation

To benefit from configuration builder support you only must add the corresponding dependency to your module:

<dependency>
  <groupId>org.apache.tamaya.ext</groupId>
  <artifactId>tamaya-model</artifactId>
  <version>{tamaya_version}</version>
</dependency>

Describing the Configuration Meta-Model

Basically configuration is modelled using key, value-pairs. Looking at a keys a.b.c.param1 and a.b.c.param2 the following concepts can be used to defined/describe configuration:

  1. the configuration section: In our case this equals to a.b.c, which itself also includes the transitive entries a.b and a.

  2. the configuration parameter: Basically parameters are adressed using their fully qualified names, which equals to the containing section name and the relative parameter name, separated by the dor separator. In the above example a.b.c.param1 and a.b.c.param2 are the fully qualified parameter names.

Now with only these 2 concepts a simple configuration meta-model can be defined as

  • a meta-model’s name, used just for grouping different meta-models and entries to better separate descriptions, e.g. in a management console or generated configuration documentation site.

  • a set of sections.

  • a set of parameters.

  • Both, sections (.model.target=Section) as well as parameter models (.model.target=Parameter)

    • can be modelled by a meta-data entry, by default _my.config.key.model.

    • may be required, or optional (.model.required=true|false)

    • may have an optional description

  • Parameters additionally have

    • a type (.model.type=Classname), described by the fully qualified class name, into which any configured (String) value must be convertable into. If no type is configured java.ui.lang.String is assumed as default.

    • an optional regular expression that can be used to validate the String values returned from a configuration (.model.expression=regexpression).

Given these concepts a configuration can be fully described. Entries that are not contained in one of the given sections (or its children), or parameters not described or marked as valid (e.g. for dynamic configModels of a section), are called undefined. Undefined parameters should be grouped with its parent section. Each section, as well as all parent sections, including transitive) of any parametet read, should similarly marked as undefined, if and only if

  1. the section itself is (directly) undefined

  2. the section is not a super section of a defined section.

As en example the section definition of a.b.c also implicitly includes the sections a.b and a to be defined sections, despite the fact that section properties, such as description and custom configModels are not inherited to its parent, or child section.

Defining Meta-Configuration Model

The configuration meta-model is defined by simple configuration meta-data entries. The section for all model configuration by default is called model, which results in entries such as _my.config.key.model.target=Section. Within this section fully qualified configuration keys defines which part of the configuration is targeted by certain entries.

Defining Sections

First we start to define some configuration sections, the example below starts with the most important variants supported:

# Metamodel information
_model.provider=ConfigModel Extension

# org.mycompany.root (optional section)
_org.mycompany.root.model.target=Section
_org.mycompany.root.model.description=Root section defining my company configuration.

# org.mycompany.security (required section)
_org.mycompany.security.model.target=Section
_org.mycompany.security.model.required=true
_org.mycompany.security.model.description=Security related settings.\
         refer for further details to XXX.

# minmal section
_minimal.model.target=Section

# custom validated section
_validated.model.target=Section
_validated.model.validator=org.apache.tamaya.model.TestValidator

Above org.mycompany.root transitively defines 3 sections:

  • org

  • org.mycompany

  • org.mycompany.root

All sections are optional. Additionally the model above also defines a required section org.mycompany.security. Required sections are checked so the section is not empty. It is not checked for any specific parameter hereby, only the existance of any child parameter is validated.

The class attribute has to be defined for any section definition, because if not set a model entry is, by default, defined to be a parameter configModel entry. Given above the entry for the section minimal shows such a minimal entry.

validated defines a section, which is validated through a customizable validator. Hereby an ordered list of validators can be provided, separated by commas.

Defining Parameters

Similarly parameters also can be defined:

# org.mycompany.root.name (required parameter)
_org.mycompany.root.name.model.target=Parameter
_org.mycompany.root.name.model.required=true
_org.mycompany.root.name.model.description=The company's name, also used in the application's main header.

# org.mycompany.security (required parameters)
_org.mycompany.security.uid.model.required=true
_org.mycompany.security.uid.model.description=The user id.
_org.mycompany.security.realm.model.required=true
_org.mycompany.security.realm.model.validator=org.apache.tamaya.model.RealmValidator
_org.mycompany.security.realm.model.description=The security realm required.
_org.mycompany.security.tokenid.model.description=The token id, if the token service is used (optional).

# A minmal parameter
_minimalClass.model.target=Class

Similarly as when defining section also parameter entries define transitively its containing sections. E.g. the entry above for org.mycompany.security.realm also defines the following sections (as optional).

  • org

  • org.mycompany

  • org.mycompany.security

Additional entries for section, e.g. configModels to be done, can be added as described in the previous section, but are optional.

Since the parameter is the default type for model entries, a minmal parameter model entry only only needs it’s parameter type to be defined. In the example above we define a parameter minimalClass of type Class. Types hereby are fully qualified class names, whereas as 'java.ui.lang' for built-in language types can be ommitted.

Model Locations

By default the configuration model can be defined at the following locations:

  • classpath*:META-INF/configmodel.properties, separate to the current Configuration. This functionality is enabled by default, but can be disabled by adding org.apache.tamaya.model.default.enabled=false to your current Configuration.

  • implicitly as part of the current +Configuration. THis can be disabled by setting the org.apache.tamaya.model.integrated.enabled configuration poarameter to false.

  • customized by configuring the org.apache.tamaya.model.resources in the current Configuration. This parameter allows to define the locations from where the model extension is trying to read the model configuration. If the resources extension is available in your system it is used to evaluate the locations. If not the default Classloader.getResources command is issued. Also it is required that the formats extension is available, since this is used to effectively read the data. This extension also allows you to use alternate representation formats such as ini, xml, yml, json.

Tracking Configuration Access

The model module also allows tracking which code accesses configuration properties or configuration parameters. It checks the stacktrace to evaluate the calling code location, hereby any unwanted packages can be implicitly ommitted from the stacktrace. Also the maximal length of the stacktrace retained can be constraint in length. The usages are recorded as Usage instances. Hereby for each parameter accessed a corresponding Usage instance is created. It can be accessed by calling Usage ConfigUsageStats.getUsage(String key). Usage statistics for calling Configuration.getProperties() can be obtained calling Usage getUsageAllProps();.

Usage tracking is disabled by default. It can be enabled by calling ConfigUsageStats.enableUsageTracking(true);. ConfigUsageStats.isUsageTrackingEnabled() returns the current tracking status.

The Usage class itself provides access to further fainer grained usage data (AccessDetail) containing:

  • the access point (fqn.ClassName#method(line: xxx)).

  • the number of accesses

  • the first an last access

  • the values read

  • the access stacktrace (filtered by ignored packages).

public final class Usage {
    [...]
    public String getKey();
    public void clearMetrics();
    public int getReferenceCount();
    public int getUsageCount();
    public Collection<AccessDetail> getAccessDetails(Class type);
    public Collection<AccessDetail> getAccessDetails(Package pack);
    public Collection<AccessDetail> getAccessDetails(String lookupExpression);
    public Collection<AccessDetail> getAccessDetails();
    public void trackUsage(String value);
    public void trackUsage(String value, int maxTraceLength);


    public static final class AccessDetail {
        [...]
        public void clearStats();
        public long trackAccess(String value);
        public long getAccessCount();
        public String getAccessPoint();
        public long getFirstAccessTS();
        public long getLastAccessTS();
        public String[] getStackTrace();
        public Map<Long, String> getTrackedValues();
    }

}

With ConfigUsageStats.clearUsageStats() the collected statistics can be reset at any time. Summarizing the main singleton for configuration statistics is defined as follows:

public final class ConfigUsageStats{
    public static Set<String> getIgnoredUsagePackages();
    public static void addIgnoredUsagePackages(String... packageName);
    public static void enableUsageTracking(boolean enabled);
    public static Usage getUsage(String key);
    public static Collection<Usage> getUsages();
    public static void clearUsageStats();
    public static Usage getUsageAllProperties();
    public static boolean isUsageTrackingEnabled();
    public static String getUsageInfo();
}

Customizing the Stacktrage for Usage Reporting

The stacktrace tracked by the system can be customized in several ways:

  • ConfigUsageStats.addIgnoredPackageNames(String...) allows to add additional ignored package names.

  • With Usage.setMaxTraceLength(int) the maximal size of the stacktraces logged can be set. Setting a negative value will disable stacktrace logging completelely.

Accessing Usage Statistics

Bascially usage statistics are available in two forms:

  • The Usage/AccessDetail object tree can be accessed programmatically from the ConfigUsageStats singleton.

  • With ConfigUsageStats.getUsageInfo() also a textual representation of the usage statistics can be obtained, as illustrated below (a snipped from the current test output):

Apache Tamaya Configuration Usage Metrics
=========================================
DATE: Sat Apr 30 21:51:09 CEST 2016

220    <<all>>:
  - 220   <unknown/filtered/internal>                       , first=Sat Apr 30 21:51:09 CEST 2016, last=Sat Apr 30 21:51:09 CEST 2016
3      java.version:
  - 2     test.model.TestConfigAccessor#readProperty(line:43), first=Sat Apr 30 21:51:09 CEST 2016, last=Sat Apr 30 21:51:09 CEST 2016
  - 1     <unknown/filtered/internal>                       , first=Sat Apr 30 21:51:09 CEST 2016, last=Sat Apr 30 21:51:09 CEST 2016

Programmatic API

Basically the configModel module provides a simple API to access the defined ConfigModel instances and validating the current Configuration against the models as follows:

public final class ConfigModelManager {

    private ConfigModelManager() {}

    public static Collection<ConfigModel> getModels();
    public static Collection<ConfigModel> findModels(ModelType type, String namePattern);
    public static <T extends ConfigModel> T getModel(String name, Class<T> modelType);
    public static Collection<ConfigModel> findModels(String namePattern);

    public static Collection<ValidationResult> validate();
    public static Collection<ValidationResult> validate(boolean showUndefined);
    public static Collection<ValidationResult> validate(Configuration config);
    public static Collection<ValidationResult> validate(Configuration config, boolean showUndefined);

    public static void registerMBean();
    public static void registerMBean(String context);

}

This singleton allows to validate the current or any Configuration instance. All the ConfigModels read also are available from the getModels method. This models can be used to provide documentation, e.g. as part of a CLI interface or shown on a documentation web server.

A ConfigModel hereby is defined as one single part of configuration, typically corresponding to a specific concern of your system. As an example you can define different models for different modules or products plugged together. With resolution mechanism in place you can also define a shared module that is targeted by multiple modules as a single configuration source (e.g. for configuring the machine’s IP address and subnet settings only once.

public interface ConfigModel {

    ModelTarget getType();
    String getName();
    String getProvider();
    boolean isRequired();
    String getDescription();
    Collection<ValidationResult> validate(Configuration config);
}

Hereby ModelTarget defines more details on the kind of model:

public enum ModelTarget {
    /**
     * A configuration section.
     */
    Section,
    /**
     * A configuration paramter.
     */
    Parameter,
    /**
     * ConfigModel that is a container of other validations.
     */
    Group
}

A ValidationResult models one validation executed by a ConfigModel on a certain Configuration instance:

public final class ValidationResult {

    public static ValidationResult ofValid(ConfigModel configModel);
    public static ValidationResult ofMissing(ConfigModel configModel);
    public static ValidationResult ofMissing(ConfigModel configModel, String message);
    public static ValidationResult ofError(ConfigModel configModel, String error);
    public static ValidationResult ofWarning(ConfigModel configModel, String warning);
    public static ValidationResult ofDeprecated(ConfigModel configModel, String alternateUsage);
    public static ValidationResult ofDeprecated(ConfigModel configModel);
    public static ValidationResult ofUndefined(final String key);
    public static ValidationResult of(ConfigModel configModel, ValidationState result, String message);

    public ConfigModel getConfigModel();
    public ValidationState getResult();
    public String getMessage(),
}

The result of a complete validation on a concrete Configuration instance finally is mapped as a Collection<ValidationResult>, refer to the methods on ConfigModelManager.

Auto-Documentation of Classes with Configuration Injection

A special feature of this module is that it observes ConfigEvent published through Tamaya’as event channel (tamaya-events module). If no metaconfiguration model is found the model manager by default automatically creates models for all injected instances on the fly. In the case of CDI integration this happens typically during deployment time, since CDI initializes during deployment time. Other runtime platforms, such as OSGI, may have rather different behaviour. Nevertheless this means that after your system has been started you should have access to a complete set of ConfigModel instances that automatically document all the classes in your system that consume configuration (through injection).

Model SPI

Registering Configuration Models

The model extension also provides an SPI where customized functionality can be added. The main abstraction hereby is the ModelProviderSpi interface, which allows any kind of additional config models to be added to the system:

public interface ModelProviderSpi {

    Collection<ConfigModel> getConfigModels();

}

New instances implementing this interface must be registered into the current ServiceContext, by default the ServiceLoader is used.

The ConfigUsageStatsSpi

The methods for managing and tracking of configuration changes are similarly delegated to an implementation of the org.apache.tamaya.model.spi.ConfigUsageStatsSpi SPI. By implementing this SPI and registerting it with the ServiceContext the usage tracking logic can be adapted or replaced.

Other Utility Classes

The module also provides further utility classes that may be useful for implementing models or testing:

  • AbstractModel provides a base class that can be extended, when implementing ConfigModel.

  • AreaConfigModel provides a ConfigModel implementation (with a corresponding Builder) to model the requirement of certain configuration sections being present, or opionally present, in the model.

  • ParameterModel provides an implementation base class for validating parameters on existence and compliance with a regular expression.

  • ConfigDocumentationMBean is the MBean registered that models similar functionality as ConfigModelManager.

  • ConfigModelGroup provides a ConfigModel that groups several child models.

  • ConfigModelReader allows to read ConfigModels from properties files as described at the beginning of this document.

Switches to enable/disable functionality

The model module provides different switches that can be used to activate or deactivate features:

  • tamaya.model.integrated.enabled allows to deactivate reading inline metaconfiguration delivered with the normal Tamaya Configuration. By default inline entries (_.abcd.model.*) are evaluated.

  • tamaya.model.default.enabled allows to deactivate reading metamodel information from classpath:META-INF/configmodel.properties. By default it is active.

  • tamaya.model.resources allows to define additional resources (loaded through the resources extension), that can be used to read metamodel information in any format using Tamaya’s format module.

  • the system property tamaya.model.autoModelEvents allows to activate/deactivate the automatic documentation of classes configured and published by Tamaya ConfiguredType event instances (e.g. published by Tamaya’s injection modules).