Aotearoa 2 is now available.
* metadata.yml is now optional
You are no longer required to define metadata for all your tokens. AO2 will generate default metadata for you (that can be used to provide command-line prompts for values, or for use by tools such as web form generator plugins).
* Inline properties
Properties such as default values, or instructions to encrypt values, can now be specified directly in your templates (rather than needing to be specified in a metadata.yml).
* Improved Maven compatibility
Using the -b (or --buildsafe) option, AO2 can now be executed as part of an application's Maven build process without causing Maven to prematurely exit.
NOTE:
AO2 is not fully backwards-compatible with AO1. However, in the updated documentation below, we point out exactly where AO2 differs from AO1 and—where possible—provide simple fixes that can be made to your existing scripts so you can leverage AO2, while continuing to use your existing templates, metadata, and values files.
AOtearoa is a command line tool that allows Application Operations teams to generate complex configurations for an application while having little to no knowledge of the application's configuration system beyond the few values they are entrusted to provide (e.g.: production credentials, certificates, private keys, etc...).
Updated for AO2
AOtearoa starts with template files. A template is a normal text configuration file that contains 'tokens' in which values will be inserted. Some examples are shown below:
# JWT signature public certificates. kid and certificate path mappings.
jwt:
clockSkewInSeconds: 60
certificate:
'100': ${oauth_certificate}
...
# Enable JWT verification flag.
enableVerifyJwt: ${jwt_security|true}
...PostgresDataSource:
DriverClassName: org.postgresql.ds.PGSimpleDataSource
jdbcUrl: jdbc:postgresql://${database_urls|out:comma-separated}/users?ssl=true&sslfactory=com.biz.SSLSocketFactory
......
tableauPassword: ${tableau_password|out:encrypt}
...Let's take a look at the tokens in these examples.
${oauth_certificate}
This is a straight-forward token. You can specify the replacement value for this in a values file, and (optionally) provide additional instructions to AO2 on how to handle this token in a metadata file (we will describe what values and metadata files are below).
${jwt_security|true}
In this case, we are providing both the token name, and providing inline instructions to AO2 on how to handle this token.
The token name is jwt_security. The text following the | character is a default value. This means if AO2 does not find a value for the token jwt_security in the values file, it will inject the value true.
${database_urls|out:comma-separated}
As with the previous example, we are providing both the token name, and providing inline instructions to AO2 on how to handle the token.
In this case, we are specifying how we want AO2 to transform the value before injecting it into the template. The out: part specifies that this is an output transformation instruction, and the comma-separated portion specifies that we want to convert a list of values into a string of comma separated values before we inject it into the template.
We will go into full detail on all transformation options below.
${tableau_password|out:encrypt}
In this example, the token name is tableau_password, and we have specified that we want to encrypt our value before injecting it into our template.
Below we can see two example values files. One for use in development, and one for use in production.
jwt_security:
- false
database_urls:
- dev-postgres-01.biz.com:9000
- dev-postgres-02.biz.com:9000
oauth_certificate:
- oauth-dev.pem
tableau_password:
- devpasswordjwt_security:
- true
database_urls:
- tss-postgres-01.biz.com:7373
- tss-postgres-02.biz.com:7373
- tss-postgres-03.biz.com:7390
oauth_certificate:
- oauth.pem
tableau_password:
- password As you can see in the examples above, you can use different value files for different environments.
Updated for AO2
UPDATE NOTE:
In AO1, the default regex pattern was{{my-token}}. If you have templates using this older default pattern, then you will need to add the--regex "\{\{(.*?)\}\}"option to those scripts.
Tokens are named placeholders that are put in template files for the purpose of being replaced with other values.
In AO2, the simplest tokens have the following form
${token_name}
where token_name is the name of the token.
Although, by default, AOtearoa 2 will look for tokens of the form ${token_name}, AO2 can find tokens of any form by allowing you to specify the regex pattern it searches for. For example, to search for tokens of the form {{my-token}}, you can use the regex option on the command line as follows: --regex "\{\{(.*?)\}\}".
New for AO2
If AO2 cannot find a value to replace a token with, it will check to see if a default value has been defined. E.g.:
${token_name|defaultValue}
or
${token_name|"defaultValue"}
Note: You can also specify default values in a metadata file.
You can also specify a transformation to be performed on a value prior to that value being injected into a template.
There are four transformations available:
copycomma-separatedencryptdecrypt
(Please see Transforms in the Metadata section below, for more details on what each of these transformations does).
You can specify a transformation as follows:
${token_name|out:<transformation-name>}
E.g.: To encrypt a value before injecting it, would be:
${token_name|out:encrypt}
If no transform is specified, then the copy transform will be used (which simply injects a value verbatim).
Again, transforms can also be specified in a metadata file.
Multiple properties can be specified in a token. One simply needs to separate them with a semicolon. E.g.:
${token_name|out:encrypt;defaultVal}
${token_name|"my default"; out:decrypt}
Updated for AO2
UPDATE NOTE 1:
In AO2, a metadata file is no longer required, though it can still be used to provide metadata for tokens. In AO2, if a value exists that has no corresponding metadata, then default metadata information will be created for that value. Inline metadata found in template files can also be used to customize the default-generated metadata.However, to fully customize the metadata for a token, you may still need to define it in a metadata file (since, not all of the properties available in a metadata file can be specified inline).
UPDATE NOTE 2:
If you choose to use a metadata file (e.g.: in order to specify properties that you cannot specify inline), then note that you no longer need to provide afilesproperty. In fact, AO2 ignores anyfilesvalues it finds in legacy (AO1) metadata files.Instead, you must specify the extensions of template files you wish AO2 to automatically check for tokens using the
-xoption on the command line. E.g.:-x yml -x xml.
In addition to templates and values, you can provide a metadata file that describes the values to be injected.
variables:
jwt_security:
min: 1
max: 1
output:
- copy
type: string
prompt-text: Enable 'true', or disable 'false' JWT verification security
defaults:
- true
database_urls:
min: 1
max: inf
output:
- comma-separated
type: string
prompt-text: Provide a list of one or more urls for the database
oauth_certificate:
min: 1
max: 1
output:
- copy
type: string
prompt-text: Enter the name of the OAuth (Auth Server) certificate
defaults:
- cert.pem
tableau_password:
min: 1
max: 1
output:
- encrypt
type: string
prompt-text: Tableau password (this will be encrypted)The metadata file allows the input to be transformed. In the current version, 4 types of transforms are supported:
copycomma-separatedencryptdecrypt
The copy transform simply pastes the value in as-is. (This is the default transform for inline properties)
The encrypt transform will encrypt the password using AES encryption. The encryption secret can be set on the command line using the -y or -key option. If an encryption secret is not specified on the command line, then the default Light4j secret will be used. E.g.: The tableau password (set as password above) will appear as CRYPT:iR7QkKyNDJ4wTRn9BEXROgm2iAkajmIv2ZR5VOsxMqs= in secret.yml.
The decrypt transform will decrypt the password using AES encryption. The decryption secret can be set on the command line using the -y or -key option. If a decryption secret is not specified on the command line, the default Light4j secret will be used to decrypt values.
Note: The encryption and decryption key are the same, since AES is a symmetric-key algorithm.
The comma-separated transform concatenates elements in the list and separates them with commas. E.g.: the list
- value1
- value2
- value3will be converted to value1,value2,value3 prior to being inserted.
Also note that you can set default values for tokens in the metadata yaml. These will show up in prompts and web forms, and will be used in cases where values are not entered in a values file.
If there is no metadata entry for a particular value or token, the AO2 will create one. E.g.: The default metadata for a token called MY_TOKEN would be:
MY_TOKEN:
min: 1
max: 1
output:
- copy
type: string
prompt-text: Please enter the value for MY_TOKENAO2 will
- Use metadata for a token from a metadata file, if it exists.
- Otherwise, it will use metadata defined inline for a token, if it exists.
- Otherwise, it will generate default metadata.
There are numerous ways in which values can be specified to the template resolver.
Below see our example production value file.
jwt_security:
- true
database_urls:
- tss-postgres-01.biz.com:7373
- tss-postgres-02.biz.com:7373
- tss-postgres-03.biz.com:7390
oauth_certificate:
- oauth.pem
tableau_password:
- passwordAll values must be entered as elements of a yaml list (even if there is only a single entry).
If a variable is specified in the metadata yaml but has no value in the values file, then this will cause an error unless either
- a default value is specified in the metadata yaml, or
- you specify
--promptson the command line, to instruct AO2 to request a value from the user on the command line
(Note that, specifying a default value inline will not prevent an error in this case, since the metadata defined in a metada file takes precedence over metadata defined inline).
The value injected into a token can come from the default value specified in a metadata file, or as an inline token property.
If command line prompts are chosen as a method to input values (using the --prompts option), then on the command line, the user would see the following:
Type \\q to quit. Enter a blank line to use default (if default exists)
[Enable 'true', or disable 'false' JWT verification security : (true)]: ⏎
>> value selected is 'true'
Next entry requires between 1 and inf values. Enter \\n to complete.
[Provide a list of one or more urls for the database]:
[Value 1]: tss-postgres-01.biz.com:7373
[Value 2]: tss-postgres-02.biz.com:7373
[Value 3]: tss-postgres-03.biz.com:7390
[Value 4]: \\n
[Enter the name of the OAuth (Auth Server) certificate]: oauth.pem
[Tableau password (this will be encrypted)]: password
Resolving templates...done.
As you can see, the prompts that are shown are specified in the metadata file (or by auto-generated metadata), and the defaults that are shown are also specified in the metadata file.
You may also use environment variables as a way to specify variable values. E.g.: prior to running a test, you may wish to temporarily turn off security in your configurations, even though your values file has jwt_security set to 'true':
$ export jwt_security=false
To enable values to be read from environment variables, you use the --env option on the command line.
You can also use a value file and environment variables, where environment variables can be used to override value file settings to enable you to do testing (e.g.: override security to a 'false' setting, or turn the logging level to 'debug' temporarily) without modifying your production values file.
This will prevent you from having to set environment variables for all your settings, and only specify overrides as environment variables.
Not yet implemented.
You can select the verbosity of output that AO2 generates. There are 5 levels that exist: TRACE, DEBUG, WARN, INFO, and QUIET. DEBUG messages are low level operational messages, whereas INFO messages are higher level status messages.
On the most verbose extreme, a log level of TRACE will output messages of all levels. On the other end, a log level of QUIET will supress all output except for the ERROR messages that are displayed if AOtearoa 2 is forced to exit.
The table below shows what types of message are outputted for each log level.
| Log Level | Trace | Debug | Warn | Info |
| Trace (t) | Y | Y | Y | Y |
| Debug (d) | N | Y | Y | Y |
| Warn (w) | N | N | Y | Y |
| Info (i) | N | N | N | Y |
| Quiet (q) | N | N | N | N |
The command line switch for log levels is -l (or equivalently --loglevel), and the switch values are listed below:
| Log Level | Value |
| Trace | t |
| Debug | d |
| Warn | w |
| Info | i |
| Quiet | q (or no value) |
The default log level is DEBUG (this is used when no switch is used). QUIET can be specified by either -l q, --loglevel q, or by simply -l (with no value) or --loglevel (with no value).
To run AOtearoa, you would run
$ java -jar ao.jar [arguments...]
with the appropriate arguments:
| Command | Description |
| -h, --help | Display help/usage information |
| -m, --metadata | Variable metadata file (optional) |
| -o, --outputdir | Output directory (required) |
| -v, --values | Value file (optional) |
| -p, --prompts | Use command line prompts to enter values (optional) |
| -e, --env | Get values from environment variables (optional) |
| -s, --server | Config Server URL (optional) |
| -r, --regex | regex pattern (optional - default is '\{\{(.*?)\}\}' ) |
| -t, --templates | Template file folder (required) |
| -x, --extension | Template file extensions (required) |
| -k, --kmetadata | Keystore metadata template (optional) |
| -y, --key | Symmetric encryption/decryption secret (optional) |
| -b, --buildsafe | Allows AO2 to run as part of a Maven build process (optional) |
| -l, --loglevel | Sets the log level (optional) |
Some examples are below:
$ java -jar ao.jar \
-m metadata.yml \ # metadata file
-t templates/ \ # template files directory
-x xml -x yml \ # injects into xml and yml files
-o config-out/ \ # output folder
-v values.yml \ # values file
-l
$ java -jar ao.jar \
-t templates/ \ # template files directory
-x xml \ # injects into xml files
-x yml \ # injects into yml files
-o config-out/ \ # output folder
-v values.yml \ # values file
--key the_secret # the AES encryption/decryption secret
$ java -jar ao.jar \
-t templates/ \ # template files directory
-x xml -x yml \ # injects into xml and yml files
-m metadata.yml \ # metadata file
-o config-out/ \ # output folder
-p # values from prompts
$ java -jar ao.jar \
-t templates/ \ # template files directory
-x sh -x yml \ # injects into bash and yml files
-m metadata.yml \ # metadata file
-o config-out/ \ # output folder
-e \ # values from environment variables
-loglevel w # log level set to WARN
$ java -jar ao.jar \
-t templates/ \ # template files directory
-x sh -x yml \ # injects into bash and yml files
-m metadata.yml \ # metadata file
-o config-out/ \ # output folder
-e \ # values from environment variables
-loglevel t # log level set to TRACE
$ java -jar ao.jar \
-t templates/ \ # template files directory
-x xml -x yml \ # injects into xml and yml files
-o config-out/ \ # output folder
-v values.yml \ # value file
-e # with environment variable overrides
$ java -jar ao.jar \
-t templates/ \ # template files directory
-x xml -x yml \ # injects into xml and yml files
-v values.yml \ # value file
-o config-out/ \ # output folder
--key the_secret \ # the AES encryption/decryption secret
-b # Maven safe-mode enabled
AOtearoa can also simplify the creation of JKS keystores.
certificates:
tableau-webserver-cert: # certificate-reference name
format: pem # certificate format
prompt-text: Tableau webserver public certificate # prompt text
filename: tableaucert.pem # filename of certificate
postgres-database-cert: # certificate-reference name
format: pem # certificate format
prompt-text: Postgres self-signed certificate # prompt text
filename: postgrescert.pem # filename of certificate
keystores:
server.keystore: # this will be the output keystore filename
base-keystore-filename: idkeystore.jks # base keystore filename
keystore-password: ${KEYSTORE_PASS|out:encrypt} # keystore password
certificates: # list of certificates to load into keystore
client.truststore: # this will be the output keystore filename
keystore-password: ${TRUSTSTORE_PASS|out:encrypt} # keystore password
certificates: # list of certificates to load into keystore
tableau-webserver-cert: tableau-uat # certificate-reference : alias
postgres-database-cert: postgres-uat # certificate-reference : aliasThe YAML required to build keystores contains two main sections: a certificates section and a keystores section.
The main certificates section describes the certificates to be loaded into the keystores that are defined in the main keystores section. For each certificate, you will need to provide a cert reference name in order to reference the certificate in the keystores section. Beyond that, you also need to provide the cert's format (only pem is currently supported), prompt text, and the cert's file name. The certificate must reside in the template folder.
For each keystore, the first item you specify is the keystore file name. This will be the file name of the keystore in output folder.
base-keystore-filename is an optional value that can be set if you wish to load certs into an existing keystore rather than create a new keystore from scratch. The existing keystore must reside in the template folder.
This is the password to the keystore. If you are creating a new keystore, then this will be the password assigned to the keystore that is created. However, if you have specified that you want to load certs into an existing keystore (by using the base-keystore-filename option), then this password must be the password of the existing keystore.
This subsection contains the (zero or more) certs you want loaded into the keystore. If you do not enter any certs in this section, AOtearoa will create an empty keystore for you.
For each cert that you do include, each line must contain the reference name of a certificate that has been defined in the main certificates section of the yaml, and the alias you want assigned to the certificate within the keystore. The certificate reference name and its alias are separated by a ':' character.
In the example shown above, you'll notice the existence of tokens throughout. Since the keystore yaml file is processed after values are read (either from user prompts, from a value file, web form, config server etc...), these values can be included in the keystore yaml. These values will be injected into the keystore yaml prior to processing of the keystore yaml file. Tokens can appear anywhere in the keystore yaml and are totally optional.
$ java -jar ao.jar \
-t templates/ \ # template files directory
-k keystore.yml \ # keystore template file
-v values.yml \ # values file
-o config-out/ # output folder
During processing, AO2 will inject values into the Keystore file (assuming it contains tokens), it will then copy the Keystore file into the template folder.This means that the template folder cannot contain a file with the same (case insensitive) name as the keystore file. AO2 will generate an error if it sees a file name conflict.
New for AO2
Using the -b (or --buildsafe) option, AO2 can be executed as part of an application's Maven build process without causing Maven to prematurely exit.
While in the examples provided, we separate the metadata and keystore files, it is actually possible to combine these files into one. For example, if you were to combine metadata and keystore configuration into a single file called _combined.yml, then you could use the following command line.
$ java -jar ao.jar \
-t templates/ \ # template files directory
-v values.yml \ # values file
-o config-out/ \ # output folder
-m combined.yml \ # metadata file
-k combined.yml # keystore template file
Note: Value files cannot be combined into a single file with metadata and keystore configuration.