# Rsyslog - oVirt Support

## Background

In the logging role context, oVirt is a special input type which handles inputs from the remote RHV systems and the files input (imfile) on the target host. The log messages are formatted in the ovirt input and forwarded to the elasticsearch. Plus, in the stable-0.1 version, it depended on the default rsyslog.conf which was deployed by setting the rsyslog_default variable to true. 

The goal of the task "RHELPLAN-42379 - Logging - Support oVirt input" is providing the same logging experience using the logging_inputs, logging_outputs, and logging_flows configuration. Then, the rsyslog_default flag (with etc/rsyslog.conf.j2 template) and the send-target-only config file are safely dropped.

## Implementation Changes

Note: in the effort of "RHELPLAN-41572 - Logging - Convert each subrole to a pair of task/var files", ovirt is also converted from a subrole to a pair of task/var files in roles/rsyslog/{tasks,vars}/inputs.

### ovirt task

This is an example of ovirt_input sends the log messages to the elasticsearch_output.
```
logging_outputs:
  - name: elasticsearch_output
    type: elasticsearch
    server_host: logging-es
    server_port: 9200
    index_prefix: project.
    input_type: ovirt
    retryfailures: false
    ca_cert: "/etc/rsyslog.d/es-ca.crt"
    cert: "/etc/rsyslog.d/es-cert.pem"
    key: "/etc/rsyslog.d/es-key.pem"
logging_inputs:
  - name: ovirt_collectd
    type: ovirt
    subtype: collectd
  - name: ovirt_engine
    type: ovirt
    subtype: engine
  - name: ovirt_vdsm
    type: ovirt
    subtype: vdsm
logging_flows:
  - name: flow_0
    inputs: [ovirt_collectd, ovirt_engine, ovirt_vdsm]
    outputs: [elasticsearch_output]
```
The ovirt logging_inputs are divided into 3 subtypes, `collectd`, `engine` and `vdsm`. They are separately configured to allow each subtype send its log messages to the different logging_outputs using logging_flows. The configuration files are generated by the ovirt task [Create oVirt input configuration files in /etc/rsyslog.](../roles/rsyslog/tasks/inputs/ovirt/main.yml) using [ovirt_input_template.j2](../roles/rsyslog/templates/ovirt_input_template.j2).

These are the snippet of the generated configuration files.
```
==> 90-input-ovirt-ovirt_collectd.conf <==
input (name="ovirt_collectd" type="imtcp" port="44514")
if $inputname == "ovirt_collectd" then {
    set $!original_raw_message = $msg;
    action(name="collectd_mmjsonparse" type="mmjsonparse" cookie="") # parse entire message as json

==> 90-input-ovirt-ovirt_engine.conf <==
input(type="imfile" file="/var/log/ovirt-engine/engine.log" tag="ovirt_engine"
      reopenOnTruncate="on" startmsg.regex="^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}")
if $syslogtag == "ovirt_engine" then {
  action(name="ovirt-engine_mmnormalize" type="mmnormalize"

==> 90-input-ovirt-ovirt_vdsm.conf <==
input(type="imfile" file="/var/log/vdsm/vdsm.log" tag="ovirt_vdsm"
      reopenOnTruncate="on" startmsg.regex="^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}")
if $syslogtag == "ovirt_vdsm" then {
  action(name="ovirt-vdsm_mmnormalize" type="mmnormalize"
```
As shown in this example, each `name` in the ovirt logging_inputs is used in the 2 places.
* in the file name, the 2 digits `90` means these configuration files have the `input` type. The next `input` is a fixed prefix, `ovirt` is from the input type, which is followed by the `name` - `ovirt_collectd`, `ovirt_engine`, and `ovirt_vdsm`.
* in the input tasks, the `name` is either set to $inputname or $syslogtag through the rsyslog input call. The `imtcp` input sets $inputname to the `name` value, while the `imfile` does $syslogtag to the `tag` value.

The elasticsearch output prepares ruleset `ruleset(name="elasticsearch_output")` to catch these calls.

### oVirt parameters
#### common
* `name` - unique name
* `type` - `ovirt`
* `subtype` - one of `collectd`, `engine`, `vdsm`; if any other value is set, the section is ignored.
* `ovirt_env_name` - default to `engine`
* `ovirt_env_uuid` - default to none
#### subtype collectd
* `ovirt_collectd_port` - port listening to the collectd inputs; default to `44514`.
* `ovirt_elasticsearch_index_prefix` - index prefix for the elasticsearch; default to `project.ovirt-metrics`
#### subtype engine
* `ovirt_input_file` - input log file to read; default to `/var/log/ovirt-engine/engine.log`
* `ovirt_elasticsearch_index_prefix` - index prefix for the elasticsearch; default to `project.ovirt-logs`
* `ovirt_vds_cluster_name` - default to none
* `ovirt_engine_fqdn` - ovirt engine fqdn; default to none
#### subtype vdsm
* `ovirt_input_file` - input log file to read; default to `/var/log/vdsm/vdsm.log`
* `ovirt_elasticsearch_index_prefix` - index prefix for the elasticsearch; default to `project.ovirt-logs`
* `ovirt_vds_cluster_name` - default to none
* `ovirt_engine_fqdn` - ovirt engine fqdn; default to none

### Replacement of rsyslog_default

The functionality is replaced with the basics input + default files output combination. The basic input, the default files output, and the flow from the basic input to the default files output are added to the example in the previous section. The basic input reads the system logs via the rsyslog imjournal, then the logs are stored in the local files, e.g., /var/log/messages by the default files output.

```
logging_outputs:
  - name: default_files
    type: file
  - name: elasticsearch_output
    type: elasticsearch
    server_host: logging-es
    server_port: 9200
    index_prefix: project.
    input_type: ovirt
    retryfailures: false
    ca_cert: "/etc/rsyslog.d/es-ca.crt"
    cert: "/etc/rsyslog.d/es-cert.pem"
    key: "/etc/rsyslog.d/es-key.pem"
logging_inputs:
  - name: basic_input
    type: basics
  - name: ovirt_collectd
    type: ovirt
    subtype: collectd
  - name: ovirt_engine
    type: ovirt
    subtype: engine
  - name: ovirt_vdsm
    type: ovirt
    subtype: vdsm
logging_flows:
  - name: flow_0
    inputs: [ovirt_collectd, ovirt_engine, ovirt_vdsm]
    outputs: [elasticsearch_output]
  - name: flow_1
    inputs: [basic_input]
    outputs: [default_files]
```

### Test case

There are 2 test playbooks [tests_ovirt_elasticsearch.yml](../tests/tests_ovirt_elasticsearch.yml) and [tests_ovirt_elasticsearch_params.yml](../tests/tests_ovirt_elasticsearch_params.yml). The test playbooks configure a basic input and three ovirt inputs, a default files output and two elasticsearch outputs, and a flow from the ovirt inputs to the elasticsearch outputs and the basic input to the default files output.

The test generates these config files in /etc/rsyslog.d.
```
00-global.conf                       30-output-files-default_files.conf
05-common-defaults.conf              31-output-elasticsearch-elasticsearch_output.conf
10-input-basics-modules.conf         31-output-elasticsearch-elasticsearch_output_ops.conf
10-input-ovirt-local_modules.conf    90-input-ovirt-ovirt_collectd.conf
10-input-ovirt-main-modules.conf     90-input-ovirt-ovirt_engine.conf
10-output-elasticsearch-module.conf  90-input-ovirt-ovirt_vdsm.conf
10-output-files-modules.conf         ovirt_engine.rulebase
11-input-basics-basic_input.conf     ovirt_vdsm.rulebase
30-output-elasticsearch.conf         parse_json.rulebase
```
The caller part from the ovirt inputs and the callees in the elasticsearch outputs look like this.
```
==> 90-input-ovirt-ovirt_collectd.conf <==
<<snip>>
if
  ($inputname == "ovirt_collectd")
  then {
    call elasticsearch_output
}
if
  ($inputname == "ovirt_collectd")
  then {
    call elasticsearch_output_ops
}

==> 90-input-ovirt-ovirt_engine.conf <==
<<snip>>
if
  ($syslogtag == "ovirt_engine")
  then {
    call elasticsearch_output
}
if
  ($syslogtag == "ovirt_engine")
  then {
    call elasticsearch_output_ops
}

==> 90-input-ovirt-ovirt_vdsm.conf <==
<<snip>>
if
  ($syslogtag == "ovirt_vdsm")
  then {
    call elasticsearch_output
}
if
  ($syslogtag == "ovirt_vdsm")
  then {
    call elasticsearch_output_ops
}

==> 31-output-elasticsearch-elasticsearch_output.conf <==
ruleset(name="elasticsearch_output") {
    if (strlen($.omes) > 0) and (strlen($.omes!status) > 0) then {
        # retry case
    <<snip>>

==> 31-output-elasticsearch-elasticsearch_output_ops.conf <==
ruleset(name="elasticsearch_output_ops") {
    if (strlen($.omes) > 0) and (strlen($.omes!status) > 0) then {
        # retry case
    <<snip>>
```

### Cleaning up

- The variable rsyslog_default and roles/rsyslog/template/etc/rsyslog.conf.j2 are eliminated.
- 40-send-targets-only.conf is no longer generated.
