Almond Monitor as most software, requires a number of configuration files.
Almond Monitor consists of two services (almond and howru), both needing configuration to work properly.
The main configuration file is almond.conf, where you can store configuration values for both the almond and the howru services.
In this file the prefix api. means the configuration will be read by howru. Configuration entries prefixed almond. and scheduler. is read by the almond process.
Other entries in the file such as data. and plugins. is needed by both services.
If you wish you can put the entries relevant to Howru in a separate file named api.conf.
If you use the Almond process as a Kafka producer, you can put the Kafka producer settings in a file named kafka.conf.
It is also possible to use prefix kafka. in the standard almond.conf-file.
All possible entries in the almond.conf are specified below.
Apart from the almond.conf (split into multiple files or not), Almond will also need a file named plugins.conf where plugins it should execute to collect monitoring data are specified.
The plugins configuration file is a semicolon separated file where each line refers to a specific plugin, using the following schema:
<![CDATA[
[service_name] description; plugin args;active;interval
]]>
As for an example:
You have a plugin called check_disk in the plugin directory declared in the plugins.directory configuration described below.
The plugin takes three parameters, -w, -c and -p for warning, critical and the path to the directory you want to control. You want the plugin to respond with warning if 10% or less is unused and if 5% or less is unused you want a critical response from the plugin.
You want to check your /var/log directory every minute and have the description "Control /var/log" in the api.
Your plugin configuration for this check would look like this:
<![CDATA[
[check_var_log] Control /var/log;check_disk -w 10% -c 5% -p /var/log;1;1
]]>
Say, you want to use the same check to also control /boot and /home, but you feel it is enough to check them every 15th and every 30th minutes respectively - and your file
would look like:
<![CDATA[
[check_boot] Control /boot;check_disk -w 10% -c 5% -p /boot;1;15
[check_home] Control /home;check_disk -w 10% -c 5% -p /home;1;30
[check_var_log] Control /var/log;check_disk -w 10% -c 5% -p /var/log;1;1
]]>
Note: The second last field, active is always 1 in these examples. If you for instance want to disable a check for some reason, you give it the value 0, as an example
<![CDATA[
[check_myservice] Control MyOwnService;my_own_control_script.sh;0;1
]]>
Note: inactivating a check will not make it disappear. It will only not be executed anymore.
That means that the last known state of the check will be presented by Almond. The monitoring item will be rendered as maintenance: true by Howru.
If you are using an external IAM for authorization, some important environmental variables are stored in a file named auth_config.py located in /opt/almond/www/api.
Finally Almond can use a configuration file named memalloc.conf.
This file should consist of either the word enabled or the word disabled.
By default "disabled" is written in the file. By changing this to enabled Almond will read how much memory the program will allocate to a number of important variables at startup.
The file read on startup and is found in:
<![CDATA[
/opt/almond/memalloc.alm
]]>
Note! Do not change values in memalloc.alm without knowing the consequences!
The usercase for editing and enable memalloc.conf is that you for instance have a script that collect a lot of metrics and therefor have a longer return string than what is usually expected from a Nagios plugin.
Or if you run Almond on a huge amount of servers you might want to slim down the memory allocated by the process to optimize system resources.
Generally it is not needed to edit the memalloc.alm values.
almond.api=
Default: true
true means Almond API is enabled, false means it is disabled.
almond.certificate=
Default: none
Path to server certificate if enabling TLS for Almond API. Alternative to scheduler.certificate.
almond.enableIamAud=
Default: false
If using an external IAM, setting this value to true will make Almond look if the user credentials have the correct audience.
almond.enforceIamRoles=
Default: false
If using an external IAM, setting this value to true will make Almond check if a logged in user is member of a role that have permissions to run plugins and change settings through API calls.
almond.iamAud=
Default: None
If using an external IAM, and setting enableIamAud to true, this entry defines which audiences are allowed by Almond.
Example of usage: almond.iamAud=almond-api
almond.iamIssuer=
Default: None
If using an external IAM, this config entry should be the url to the IAM issuer.
As an example: https://url.to.provider:port/realms/almondmonitor.
almond.iamRolesAccepted=
Default: None
If using an external IAM, and setting enforceIamRoles to true, here you define which roles should have executable permissions within the Almond API.
As an example: almond-exec, admins.
almond.iamPublicKeyFile=
Default: None
If using an external IAM, you need to provide the public key from the provider.
As an example: opt/almond/iam_pubkey.pem.
almond.key=
Default: none
Path to key for server certificate if enabling TLS for Almond API. Alternative to scheduler.key.
almond.port=
Default: 9909
Port used by Almond API. Alternative to scheduler.port.
almond.pushInterval=
Default: 120
If enabling data and or metrics push, this config entry tell how often Almond will run a push. Value is set in seconds.
almond.pushPort=
Default: None
If enabling data and or metrics push, this config entry should hold the port of the endpoint.
almond.pushUrl=
Default: None
If enabling data and or metrics push, this config entry hold the url to which Almond push data, usually a HowRU proxy endpoint.
almond.standalone=
Default: false
Changing this to true will make Almond run in standalone mode.
This means it will no longer export files to Howru.
almond.useMetricsPush=
Default: false
Changing this to true will make Almond push metrics to an endpoint, ususally a HowRU proxy.
almond.usePush=
Default: false
Changing this to true will make Almond push plugin data to an endpoint, ususally a HowRU proxy.
almond.useSSL=
Default: false
Changing the value to true will force Almond API to use TLS connection. You will need to use certificates on the server and also provide valid certificates while connecting to Almond API. This entry can be used instead of scheduler.useTLS
api.activeMods=
Default: modyaml.py
If setting api.enableMods to true, here you list the mods you want to run.
Mods are basic data format converters, and you can add your own if needed. Most Almond installations ship with prebuilt mods for yaml and xml.
NOTE: It is not a good idea to have to many mods running simultaneously. The recommendation is to have no more than two.
api.adminUser=
Default: admin
Username for logon to admin page.
This will update users.conf and will be removed from config after being read.
api.adminPassword=
Default: password
Password for logon to admin page.
This will update users.conf and will be removed from config after being read.
api.authProvider=
Default: local
The standard authProvider is local, but if you have an external IAM provider it can be used for login to the admin page.
Currently supported, apart from local are the following keycloak, entra, oath0 and okta.
api.authType=
Default: basic
basic (standard) gives a HTTP Auth for logging into admin page. The other option is 2fa which will enable two-factor authorization.
api.bindPort=
Default: 80
Port used by the api.
Should preferably be 443 if you use SSL, but the API can use any port.
api.dataDir=
Default: /opt/almond/data
The directory from where the api should read (the) data file(s).
api.enableAliases=
Default: false
Changing the value to true will allow the api to uses aliases as specified in the file /etc/almond/aliases.conf..
api.enableFile=
Default: false
Changing the value to true will allow the api to read from a specific file specified by whichjson.
api.enableGUI=
Default: true
Changing the value to false will disable HTML outputs.
api.enableLoginRedirect=
Default: true
If using an external IAM, setting api.enableOath to true this config entry will redirect the admin page to the external iam provider for logins.
Setting this to value to false while enabling external IAM will require a ROPC configuration/solution on the external provider side.
api.enableMods=
Default: false
Enabling mods will make HowRU run data conversion scripts (mods) while reading monitoring data.
Almond installations comes with mods for yaml and xml, but you could write your own mods of other data formats are needed by your organisation.
api.enableOath=
Default: false
Changing this value to true will enable HowRU admin page to have an external login from an IAM provider.
api.enableOtelExporter=
Default: false
Changing this value to true will enable the HowRU OTEL exporter.
api.enableOtelFileWatcher=
Default: false
Changing this value to true will make the OTEL exporter send metrics and logs to a receiving endpoint (usually an otel collector) as soon as any data file have changes.
api.enablePeriodicOtelExport=
Default: true
If otel exporter is enabled, the periodic export send metrics and logs to the receiving endpoint at a given time intervall set by api.otelExportInterval .
NOTE: You should enable either the PeriodicOtelExport or the OtelFileWatcher. Having both enabled is not recommended.
api.enableProxyCleaner=
Default: false
If using HowRU as a proxy, by enabling the proxy cleaner data files not updated within the value given in api.proxyCleanerSeconds will be removed.
api.enableScraper=
Default: false
Changing the value to true will allow the api to make sure metrics exported as prometheus will function as an exporter.
api.enableSSL=
Default: false
Changing the value to true will make HowRU use https in stead of http.
api.executableRoles=
Default: None
This configuration entry specify which roles have access to edit plugins and run commands in the admin page gui.
Leaving it blank or omitting it will make any logged in user able to edit and run commands. Roles will also be picked up from external identity providers.
The value of the field can be either a comma separated string or an array:
Examples: api.executableRoles=almond-exec, admin, api.executableRoles=['almond-exec', 'admin'].
api.isContainer=
Default: false
If your Almond runs in a docker container you should change this value to true, else HowRU will not be able to talk to Almond API.
api.isMetricsProxy=
Default: false
Changing the value to true will allow the api to read metrics from multiple servers.
api.isProxy=
Default: false
Changing the value to true will allow the api to read output from multiple servers.
api.metricsDir=
Default: /opt/almond/data/metrics
The directory from where HowRU api will read metrics files.
api.multiMetrics=
Default: false
Same as api.isMetricsProxy.
api.multiServer=
Default: false
Same as api.isProxy.
api.otelExporterInterval=
Default: 60
If enabling the otel exporter and using PeriodicOtelExport, this configuration value tells HowRU how often (in seconds) the exporter will send metrics and logs to the receiving endpoint.
api.otlpEndpoint=
Default: None
If enabling the otel exporter this configuration entry needs to be set to the url of the receiving endpoint.
api.persistant2fa=
Default: false
Changing the value to true will allow HowRU to persistantly store Auth2FA keys.
api.proxyCleanerSeconds=
Default: 320
If proxyCleaner is enabled, the value set in this configuration is the allowed timeframe for a data file to live in seconds (if not updated) before being removed by the cleaner.
api.showDashboard=
Default: on
If set to off or false HowRU will show list in stead of dashboard in proxy mode.
api.sslCertificate=
Default: /opt/almond/www/api/ssl/cert.pem
The path to your SSL certificate. The default value is just an example.
api.sslKey=
Default: /opt/almond/www/api/ssl/key.pem
The path to your SSL key used with the certificate. The default value is just an example.
api.stateType=
Default: systemctl
This should be supervisorctl when running with supervisord in Docker containers.
Otherwise the default value should be used.
api.startPage=
Default: admin
Which page should be startpage when using http(s).
Allowed values are admin, status and metrics.
api.useGUI=
Default: true
Changing the value to falsei> will disable HowRU http pages.
api.userFile=
Default: /etc/almond/users.conf
The file where HowRu stores login credentials for the admin page.
api.useSSL=
Default: false
Changing the value to 1 will force the API to run through SSL. Make sure you have api.sslCertificate and api.sslKey values in your config.
data.jsonFile=
Default: monitor_data.json
The name of the file where the scheduler writes data and from where the api will read it.
Note: if using proxy server functionality this value must be different for the various servers.
data.metricsFile=
Default: monitor.metrics
The name of the file where the scheduler should write current metrics.
Note: if using proxy server functionality this value must be unique among the servers connected to the proxy.
Note: if using Prometheus metrics and using OpenTelemetry or likewise, change the subfix to .prom.
data.metricsOutputPrefix=
Default: almond
Prefix given to all metrics created by this instance of Almond.
data.saveOnExit=
Default: false
Changing the value to true will append data to files when restarting the application.
If api.isProxy is enabled or you run the scheduler in a container this value is recommended to be false.
gardener.CleanUpTime=
Default: 86400
Value is given in seconds.
Default value will drop all metrics files older than 24 hours when gardener runs.
plugins.declaration=
Default: /etc/almond/plugins.conf
The file where you configure how plugins should run.
plugins.directory=
Default: /opt/almond/plugins/
The directory where you store your plugins.
You can also point to for instance /usr/local/nagios/libexec/ or symlink other directories to the howru plugins directory.
scheduler.allowAllHosts=
true
If set to false Almond will require a file named /etc/almond/allowed_hosts
where you specify which servers are allowed to connect to Almond API.
scheduler.certificate=
Default: none
Path to server certificate if enabling TLS for Almond API.
scheduler.clearDataCacheInterval=
Default: 300
The interval on which Almond wil run its clearDataCash-function, if it´s enabled.
scheduler.confDir=
Default: /etc/almond/
Path to directory of configuration files.
scheduler.dataCacheTimeFrame=
Default: 330
The timeframe an unused data file is allowed to be stored in the Almond data directory.
scheduler.dataDir=
Default: /opt/almond/data/
The directory where to store output.
scheduler.enableClearDataCache=
Default: false
Changing this value to true will make Almond start a thread to clear old data files.
scheduler.enableGardener=
Default: false
If set to true Almond will trigger a gardener script to remove old files.
scheduler.enableKafkaExport=
Default: false
Changing the value to true will make Almond start its Kafka producer.
The producer will not start if not all Kafka settings are correct.
scheduler.enableKafkaId=
Default: false
Changing the value to 1 will make Almond Kafka producer add id as extra metadata to content. Works together with scheduler.kafkaStartId.
scheduler.enableKafkaSSL=
Default: false
Changing this value to true will force Almond to connect to Kafka with SSL.
scheduler.enableKafkaTag=
Default: false
If set to true you can add an additional tag to the message Almond is producing to a Kafka topic.
scheduler.format=
Default: /opt/almond/data/
Format of output. The default is json, other possible formats are metrics and the combination of the two jsonmetrics
You can also use prometheus style metrics.
scheduler.gardenerRunInterval=
Default: 43200
This parameter tell Almond how often it should run the gardener script. The default value will make it run every 12 houers.
scheduler.gardenerScript=
Default: /opt/almond/gardener.py
Path to the script you want to run to "garden" your metrics files.
scheduler.hostName=
Default: none
Howru will automatically look up hostname on the server where it´ running and this configuration is therefor optional.
However if running in a container you might want to give a virtual hostname to the exports from the container.
scheduler.initSleepMs=
Default: 7000
The sleep time in miliseconds between start of each plugin at startup.
scheduler.kafkaAvro=
Default: false
Setting this to true will make Almond use Avro when producing data to Kafka.
You need to import schema registry to Kafka and set schemaRegistryUrl, schemaName as well as kafkaStartId and kafkaTag parameters.
scheduler.kafkaBrokers=
Default: none
Url and port to Kafka broker(s)r as ip:port.
scheduler.kafkaCACertificate=
Default: none
If scheduler.enableKafkaSSL is set you need to provide the path to the CA certificate used in the SSL key chain.
scheduler.kafkaConfigFile=
Default: /etc/almond/kafka.conf
he file where you want to store configuration for Kafka producer. Read further on scheduler.useKafkaConfigFile.
scheduler.kafkaProducerCertificate=
Default: none
If scheduler.enableKafkaSSL is set you need to provide the path to the certificate that Almond should use when producing to its Kafka topic.
scheduler.kafkaStartId=
Default: 0
If scheduler.enableKafkaId is set you can choose what the first of an incremental id attached to each check made by Almond passed as a metadata to Kafka.
scheduler.kafkaTag=
Default: none
The tag that you want to add to a the data Almond produces to a Kafka topic.
scheduler.kafkaTopic=
Default: almond_monitoring
Name of the Kafka topic where you want to produce your monitoring data.
scheduler.key=
Default: none
Path to key for server certificate if enabling TLS for Almond API. Same as almnónd.key.
scheduler.logDir=
Default: /var/log/almond/
Path to log file directory.
scheduler.logPluginOutput=
Default: false
Changing the value to true will make Almond write the results of plugin runs to the standard log file.
scheduler.logToStdout=
Default: false
Changing the value to true will make Almond write verbose to stdout.
Might be interesting if it runs in a container.
scheduler.quickStart=
Default: false
Changing the value to true will force Almond to run all plugins at once before starting the scheduler.
scheduler.runGardenerAtStart=
Default: false
Changing the value to true will force Almond to run the gardener script at startup.
scheduler.schemaName=
Default: none Example: almond-montitor-topic-value
The name of the schema registry used by Almond as a Kafka producer if avro is enabled.
scheduler.schemaRegistryUrl=
Default: none
The url to where Kafka stores its schema registry (http://localhost:8081). Needed by Almond as a Kafka producer only if kafkaAvro is enabled.
scheduler.sleepMs=
Default: 5000
Sleep time for the scheduler timer. Default is looking every 5th second if it´s time to run a command.
scheduler.storeDir=
Default: /opt/almond/data/metrics
The directory where you store plugin results if enabled in the scheduler.storeResults parameter.
scheduler.storeResults=
Default: false
Changing the value to true will make Almond write the results of plugins to the directory specified in scheduler.storeDir. The value should be set to 1 if you use metrics instead of or together with json.
scheduler.truncateLog=
Default: false
If value is set to true, Almond will compress and truncate the log file at the interval set intruncateLogInterval.
scheduler.truncateLogInterval=
Default: 604800
If scheduler.trucateLog is activated, the value added to this entry will determine how much log that is going to be saved (in seconds).
scheduler.tuneCycle=
Default: 15
Value given in minutes.
How often the time tuner function (if enabled) will checking for plugin runs being scheduled to run in proximity to each other.
scheduler.tuneMaster=
Default: 1
Value given as plugin id.
The tune master could be any declared plugin whose next run timestamp is used to calculate runtimes for the scheduler. Only valid if tuneTimer is enabled.
scheduler.tuneTimer=
Default: false
Changing the value to true will activate the tuneTimer function.
The tune timer will try to prevent chunks of plugins running simultaneously. It might delay plugin runs, in order to minimize load.
scheduler.type=
Default: classic
Changing the value to time instead of classic will implement the new scheduler introduced in version 0.9.6.
scheduler.useExternal=
Default: false
Changing the value to true will disable Almond internal scheduler for executing check commands.
Almond will solely reply to calls to Almond API to execute commands. This is mostly for use with check_almond with Nagios equivelent solution.
scheduler.useKafkaConfigFile=
Default: false
Changing this value to true will make the Almond Kafka producer read all configurations from the file specified in scheduler.kafkaConfigFile.
Take note that kafka_brokers, kafka topic and SSL related information added in almond.conf will be overriden.
You can set the kafka file to almond.conf prefixing Kafka related configurations with kafka.. All Kafka producer related configurations prefixed scheduler. will be omitted.
scheduler.useTLS=
Default: false
Same as almond.useTLS.
Note! These settings can be placed in its own file kafka.conf, or if you choose can be used in the main configuration file (almond.conf.
kafka.bootstrap.servers=
Kafka producer property: bootstrap.servers
Default: none.
Must be specified
Note! scheduler.kafkaBrokers will not be read if this entry is set.
kafka.topic=
Kafka producer property: none
Default: none. Must be specified
Note! scheduler.kafkaTopic will not be read.
kafka.client.id=
Kafka producer property: client.id
Default: almond.
kafka.metadata.max.age.ms=
Kafka producer property: metadata.max.age.ms
Default: 60000.
kafka.acks=
Kafka producer property: acks
Default: 1.
kafka.enable.idempotence=
Kafka producer property: enable.idempotence
Default: false.
kafka.transactional.id=
Kafka producer property: transactional.id
Default: none. Type: String Requires: kafka.enable.idempotence
kafka.retries=
Kafka producer property: retries
Default: 5.
kafka.max.in.flight.requests.per.connection=
Kafka producer property: max.in.flight.requests.per.connection
Default: 5.
kafka.linger.ms=
Kafka producer property: linger.ms
Default: 10.
kafka.batch.num.messages=
Kafka producer property: batch.num.messages
Default: 10000.
kafka.queue.buffering.max.messages=
Kafka producer property: queue.buffering.max.messages
Default: 100000.
kafka.queue.buffering.max.kbytes=
Kafka producer property: queue.buffering.max.message
Default: 100000. Note Deprecated! Will no longer have effect.
kafka.compression.codec=
Kafka producer property: compression.codec
Default: lz4.
kafka.message.max.bytes=
Kafka producer property: message.max.byte
Default: 1000000.
kafka.message.copy.max.bytes=
Kafka producer property: message.max.byte
Default: 65535.
kafka.request.timeout.ms=
Kafka producer property: request.timeout.ms
Default: 30000.
kafka.delivery.timeout.ms=
Kafka producer property: delivery.timeout.ms
Default: 30000.
kafka.retry.backoff.ms=
Kafka producer property: retry.backoff.ms
Default: 100.
kafka.dr_cb=
Kafka producer property: none
Default: true.
kafka.statistics.interval.ms=
Kafka producer property: statistics.interval.ms
Default: 60000.
kafka.log.connection.close=
Kafka producer property: log.connection.close
Default: false.
kafka.ssl.ca.location=
Kafka producer property: ssl.ca.location
Default: none.
Replaces scheduler.kafkaCACertificate if enabling kafka config.
kafka.ssl.certificate.location=
Kafka producer property: ssl.certificate.location
Default: none.
Replaces scheduler.kafkaProducerCertificate if enabling kafka config.
kafka.ssl.key.location=
Kafka producer property: ssl.key.location
Default: none.
eplaces scheduler.kafkaSSLKey if enabling kafka config.
kafka.security.protocol=
Kafka producer property: security.protocol
Default: none.
kafka.sasl.mechanisms=
Kafka producer property: sasl.mechanisms
Default: none.
kafka.sasl.username=
Kafka producer property: sasl.username
Default: none.
kafka.sasl.password=
Kafka producer property: sasl.password
Default: none.
kafka.socket.timeout.ms=
Kafka producer property: socket.timeout.ms
Default: 600000.
kafka.transaction.timeout.ms=
Kafka producer property: transaction.timeout.ms
Default: 30000.
kafka.socket.blocking.max.ms=
Kafka producer property: socket.blocking.max.ms
Default: 10000.
kafka.partitioner=
Kafka producer property: partitioner
Default: consistent_random.
