Deploy OSM and LMA on AWS


#1

The Charmed Distribution of OSM (CDO) can be paired with Logging, Monitoring and Alerting (LMA) stack. Before deploying CDO, lets setup the LMA stack on OSM by executing the following steps:

1. Bootstrap the AWS Controller:

juju bootstrap aws aws-osm-ha

2. Create the graylog-vhost.tmpl file with the following content:

<Location "/">
    RequestHeader set X-Graylog-Server-URL "http://{{ servername }}/api/"
    ProxyPass http://{{ graylog_web }}/
    ProxyPassReverse http://{{ graylog_web }}/
</Location>

<Location "/api/">
    ProxyPass http://{{ graylog_api }}/api/
    ProxyPassReverse http://{{ graylog_api }}/api/
</Location>

3. Create the osm-lma-bundle.yml bundle file with the following content:

series: bionic
variables:
  nagios-context: &nagios-context osm-lma
applications:
  apache2:
    charm: cs:apache2
    constraints: cores=1 mem=4G
    num_units: 1
    expose: true
    options:
      enable_modules: "headers proxy_html proxy_http"
      vhost_http_template: include-base64://graylog-vhost.tmpl
  elasticsearch:
    charm: cs:elasticsearch
    num_units: 1
    constraints: cores=8 mem=32G
    options:
      firewall_enabled: False
      es-heap-size: 16
  graylog:
    charm: cs:graylog
    num_units: 1
    constraints: cores=4 mem=8G
  mongodb:
    charm: cs:mongodb
    constraints: cores=2 mem=4G
    num_units: 1
  grafana:
    charm: cs:grafana
    constraints: cores=2 mem=4G
    num_units: 1
  prometheus:
    charm: cs:prometheus2
    constraints: cores=2 mem=4G
    num_units: 1
  nagios:
    num_units: 1
    constraints: cores=2 mem=4G
    charm: cs:nagios
  nrpe-host:
    charm: cs:nrpe
    options:
      nagios_hostname_type: "host"
      nagios_host_context: *nagios-context
      xfs_errors: "30"
      swap: ''
  filebeat:
    charm: cs:filebeat
    options:
      logpath: "/var/log/*.log /var/log/*/*.log /var/log/syslog"
  telegraf:
    charm: cs:telegraf
relations:
  - [ "apache2:reverseproxy", "graylog:website" ]
  - [ "graylog:elasticsearch", "elasticsearch:client" ]
  - [ "graylog:mongodb", "mongodb:database" ]
  - [ "filebeat:logstash", "graylog:beats" ]
  - [ "prometheus:grafana-source", "grafana:grafana-source" ]
  - [ "telegraf:prometheus-client", "prometheus:target" ]
  - [ "apache2:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "elasticsearch:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "graylog:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "mongodb:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "grafana:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "prometheus:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "nagios:monitors", "nrpe-host:monitors" ]
  - [ "nagios:juju-info", "telegraf:juju-info" ]
  - [ "nagios:juju-info", "filebeat:beats-host" ]

4. Deploy the LMA bundle and offer the relationships to other models:

juju add-model lma
juju deploy osm-lma-bundle.yml lma-stack
juju offer lma.graylog:beats graylog-beats
juju offer lma.prometheus:target prometheus-target
juju offer lma.nagios:monitors nagios-monitors

5. Create the k8s-aws-overlay.yml file with following content:

cat << EOF > k8s-aws-overlay.yaml
variables:
  nagios-context: &nagios-context osm-cdk
applications:
  aws-integrator:
    charm: cs:~containers/aws-integrator
    num_units: 1
  osm-vca:
    charm: cs:~charmed-osm/vca
    num_units: 1
    constraints: mem=4G cores=2 root-disk=20G
  kubernetes-master:
    options:
      nagios_context: *nagios-context
  kubernetes-worker:
    num_units: 4
    options:
      nagios_context: *nagios-context
  filebeat:
    charm: cs:filebeat
    options:
      logpath: '/var/log/*.log /var/log/*/*.log /var/log/syslog'
  filebeat-kube:
    charm: cs:filebeat
    options:
      logpath: '/var/log/*.log /var/log/*/*.log /var/log/syslog /var/log/pods/*/*/*.log'
      kube_logs: true
  telegraf:
    charm: cs:telegraf
  nrpe-host:
    charm: cs:nrpe
    options:
      nagios_hostname_type: host
      nagios_host_context: *nagios-context
      xfs_errors: "30"
      swap: ''
relations:
  - ['aws-integrator', 'kubernetes-master']
  - ['aws-integrator', 'kubernetes-worker']
  - [ "filebeat-kube:beats-host", "kubernetes-master:juju-info" ]
  - [ "filebeat-kube:beats-host", "kubernetes-worker:juju-info" ]
  - [ "filebeat:beats-host", "easyrsa:juju-info" ]
  - [ "filebeat:beats-host", "etcd:juju-info" ]
  - [ "kubernetes-master:juju-info", "telegraf:juju-info" ]
  - [ "kubernetes-worker:juju-info", "telegraf:juju-info" ]
  - [ "easyrsa:juju-info", "nrpe-host:general-info" ]
  - [ "etcd:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "kubeapi-load-balancer:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "kubernetes-master:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "kubernetes-worker:nrpe-external-master", "nrpe-host:nrpe-external-master" ]
  - [ "aws-integrator:juju-info", "nrpe-host:general-info" ]
EOF

7. Deploy the CDK bundle with the respective overlay and relate with the LMA model:

juju add-model cdk
juju deploy charmed-kubernetes --overlay k8s-aws-overlay.yaml
juju trust aws-integrator
juju offer osm-vca:osm-vca # Offer osm-vca for a Cross-Model Relation
juju relate filebeat:logstash lma.graylog-beats
juju relate filebeat-kube:logstash lma.graylog-beats
juju relate nrpe-host:monitors lma.nagios-monitors
juju relate telegraf:prometheus-client lma.prometheus-target

8. Wait for the CDK deployment to be ready.

watch -n1 -c juju status --color

9. Deploy CDO

In order to deploy CDO, follow the OSM-HA deployment documentation (skip CDK install).