[Tutorial] Charm development (Beginner, part 1)


#1

Tutorial: Charm development (Beginner, part 1)

Difficulty: Beginner
Author: Erik Lönroth

What you will learn

This guide will go through the first basic concepts of charm development.

This is the first of a few tutorials in getting started with juju development:

[Part 1] - First steps developing juju charms
Part 2 - Adding in functionality with “layers” and connecting to a database.
Part 3 - Uploading charms to charm store.
[Part 4 (not yet written)] - Interfaces, flags & understanding the state-machine of juju.

You will learn in this part:

  • Preparing & setup of a basic workbench.
  • Creating the example charm with “charm tools”.
  • Understanding the anatomy of a charm (files and directories).
  • Validating & building the charm.
  • Adding functionality via a secondary layer (layer:apt).
  • Deploying the example charm with juju.

Setup up a basic workbench

This is how a typical workbench looks like:

  • A Juju controller: To deploy developed charms to. You can [start here][getting-started] to get one up and running.

  • Python 3.x: We use python 3 in this tutorial to develop our charm.

  • Charm Tools: To create skeleton charms, build, fetch and test charms. See the Charm Tools page
    for installation instructions.

  • Three directories for our build environment needs to be created.

    mkdir -p ~/charms
    mkdir -p ~/charms/layers
    mkdir -p ~/charms/interfaces
  • Put these environment variables in your ~/.bashrc
    export JUJU_REPOSITORY=$HOME/charms
    export LAYER_PATH=$JUJU_REPOSITORY/layers
    export INTERFACE_PATH=$JUJU_REPOSITORY/interfaces
  • Finally source your ~/.bashrc to get the environment properly setup.
    source ~/.bashrc

Creating the example charm with “charm tools”

To simplify creation of new charms, charmtools exist for us. Lets start a new charm that we name: “layer-example”.

cd ~/charms/layers
charm create layer-example

Great work, lets move on to understand what a charm consists of.

The anatomy of a charm

A bare minimum charm consists of a directory with the charm name and two files: ‘layers.yaml’ and ‘metadata.yaml’. Thats all that is strictly required for a charm to be valid. We do however normally create a directory called ‘reactive’ where we put a python module named with our charm. This is what happened when we ran ‘charm create layer-example’ above.

Lets examine what was created.

layer-example
├── config.yaml             <-- Configuration options for our charm/layer.
├── icon.svg                <-- A nice icon for our charm.
├── layer.yaml              <-- The layers and interfaces we include.
├── metadata.yaml           <-- Information about our charm
├── reactive                <-- Needed for all reactive charms
│   └── layer_example.py    <-- The charm code
├── README.ex               <-- README
└── tests                   <-- Tests goes in here
    ├── 00-setup            <-- A skeleton setup test
    └── 10-deploy           <-- A skeleton deploy test

Note! Prefixing the charm directory name with ‘layer-’ is a naming convention. It tells us that this charm is a ‘reactive’ charm. You can read even more on this topic in the official documentation here: Charms.Reactive

Validating the charm

If we were to build our charm now, it would fail because its created with defaults. We can see this, by running “charm proof” to validate our charm structure:

cd ~/charms/layers
charm proof layer-example

I: Includes template icon.svg file.
I: no hooks directory
W: no copyright file
W: Includes template README.ex file
W: README.ex includes boilerplate: Step by step instructions on using the charm:
W: README.ex includes boilerplate: You can then browse to http://ip-address to configure the service.
W: README.ex includes boilerplate: - Upstream mailing list or contact information
W: README.ex includes boilerplate: - Feel free to add things if it's useful for users
E: template interface names should be changed: interface-name
I: relation provides-relation has no hooks
E: template interface names should be changed: interface-name
I: relation requires-relation has no hooks
E: template interface names should be changed: interface-name
I: relation peer-relation has no hooks
I: missing recommended hook install
I: missing recommended hook start
I: missing recommended hook stop
I: missing recommended hook config-changed

Let’s get rid of these E: errors by making the following files look like this:

layer-example/layer.yaml

(Note: layer:basic is always included in reactive charms to provide core functionality)

includes:
  - 'layer:basic'

layer-example/metadata.yaml

name: example
summary: A very basic example charm
maintainer: Your Name <your.name@mail.com>
description: |
  This is a charm I built as part of my beginner charming tutorial.
tags:
  - misc
  - tutorials

layer-example/reactive/layer_example.py

from charms.reactive import when, when_not, set_state

@when_not('example.installed')
def install_example():
    set_flag('example.installed')

Building the example charm

We are ready to build our charm now with charm tools.

cd ~/charms/layers
charm build layer-example

build: Composing into /home/erik/charms
build: Destination charm directory: /home/erik/charms/trusty/example
build: Please add a `repo` key to your layer.yaml, with a url from which your layer can be cloned.
build: Processing layer: layer:basic
build: Processing layer: example (from layer-example)
proof: I: Includes template icon.svg file.
proof: W: Includes template README.ex file
proof: W: README.ex includes boilerplate: Step by step instructions on using the charm:
proof: W: README.ex includes boilerplate: You can then browse to http://ip-address to configure the service.
proof: W: README.ex includes boilerplate: - Upstream mailing list or contact information
proof: W: README.ex includes boilerplate: - Feel free to add things if it's useful for users
proof: I: all charms should provide at least one thing

Great work! Your charm is assembled and placed in the $JUJU_REPOSITORY/trusty/example directory. Go ahead and look in to it before we move on.

Adding functionality via a layer

Our example charm isn’t really doing anything fun yet. Let’s make it install the ‘hello’ package and set a “Hello World” message for juju once its done.

For this very common scenario of installing packages as part of a charm, we can use the layer:apt.

The layer:apt has all the functionality we need for installing packages from apt repositories. (You will learn more about including layers in the next part of the tutorial)

Modify the ‘~/charms/layers/layer-example/layer.yaml’ to look like this:

includes: 
  - 'layer:basic'
  - 'layer:apt'
options:
  apt:
    packages:
     - hello

Modify ~/charms/layers/layer-example/reactive/layer_example.py to look like this:

from charms.reactive import set_flag, when, when_not
from charmhelpers.core.hookenv import application_version_set, status_set
from charmhelpers.fetch import get_upstream_version
import subprocess as sp

@when_not('example.installed')
def install_example():
    set_flag('example.installed')

@when('apt.installed.hello')
def set_message_hello():
    # Set the upstream version of hello for juju status.
    application_version_set(get_upstream_version('hello'))

    # Run hello and get the message
    message = sp.check_output('hello', stderr=sp.STDOUT)

    # Set the active status with the message
    status_set('active', message )

    # Signal that we know the version of hello
    set_flag('hello.version.set')

Lets build again with our changes.

cd ~/charms/layers/
charm build layer-example

The charm will now be built and the final charm assemble ends up in
‘~/charms/layers/trusty/example’

Deploy it with juju:

cd ~/charms/layers/trusty/
juju deploy ./example

After some time, juju status will show the “Hello World” message.

Congratulations, you have completed the first basic exercise in charm development!

More to learn from this tutorial:

Layers vs Charms

One way of thinking about layers in relation to charms, is in terms of libraries or modules. A compilation of layers results in a charm that can be deployed by the juju engine.

There are a lot of layers included in charmtools, you can find them in the layer-index that we will cover in the next part of this tutorial series.

How to think about ‘Reactive programming’

Most programmers expects their applications to be executing from a clear “main()” start and move on step by step towards an exit. Reactive programming is ‘somewhat’ different in how you plan the execution.

In reactive programming, a good way of thinking about your program, is that it has many “main()” entry points. Which one is executed - and when - depends on how you act on the different states/flags communicated to you by the juju engine.

The principle is that juju engine signals your application, and you write code/functions to act on this information. Your code then raises new flags/states to communicate with the rest of the system.

This is what the @when(some.flag.raised) decorators are all about.

Next lesson

Building on your new knowledge, you should now move to Part 2

Contributors

I’ve deleted the conversations to keep the tutorial clean in the work

@manadart: typos
@seffyroff: Trying the tutorial out and fixes, errors.
@wallyworld: Good advices and public cheering
@rick_h: Helping out to manage the structure of tutorials


[Tutorial] Charm development (Beginner, part 2)
The Juju Show #42 - 18:00UTC - Nov 14th
[Tutorial] Charms with layer-snap (Intermediate)
Adding tutorials
#2

Thanks Eric! This is truly excellent and a much needed addition to our documentation. Reactive charms are great but as you’ve no doubt discovered there can be a learning curve to get started. But the effort certainly pays off in the long run. Thanks again for taking the time to contribute; I’m sure a lot of people will appreciate the the work here.


#4

Thanks for taking the time and effort to write this! I followed through the first part and spotted a couple of potential typos - perhaps you could drop the tutorial in a repo and I’d be happy to PR against it?


#8

hmm, I’ll keep my comments here I think - let’s not muddy the waters whilst the tutorials site isn’t playing nice for you.

Speaking of comments:
You have the student initially create 3 dirs in ~/. The interfaces dir is created here:

mkdir -p ~/charms/interfaces

In the next section you setup some env vars, and the interface path is set to:

export INTERFACE_PATH=$JUJU_REPOSITORY/interface

I suggest changing this one to:

export INTERFACE_PATH=$JUJU_REPOSITORY/interfaces

#11

I don’t know if folks noticed, but if you highlight text from a message you get this ability to “Quote” it. It helps in providing feedback to a section of the document. If you have edit ability I’d suggest just collaborating on the post. If not, I’d suggest a change and then the author or someone else can make the edit, thank the contributor, and then possibly delete/etc the suggestion to keep the conversation clean of things that are done with. There’s some room for us to figure out a workflow and such from here.


#14

Personally, I think a thorough tutorial on interfaces needs also a VERY clear, pedagogical, structured, easy-to-read, visual, description of all the ‘core-flags-and-state-machine-behaviour’. When juju starts throwing flags, its where things get bananas.

It was at this point, I got totally messed up in trying to even remotely have a implementational strategy for reactive charming.

I believe this is where we lose contributors trying to achieve true juju magic. Can’t even today honestly say I grasp it fully. Its also obvious to me, most charmers dont. It tells from the code of different charms with different authors. There is totally not a best practice or template here… I might be wrong.


#15

You are 100% correct. The material you have put together is the best doc I have seen to date on reactive charming because it takes you on a journey through the process from a development perspective. There’s other reference material out there, but you need both workflow oriented and reference doc to be productive. And I don’t know of any doc that truly covers the material you mention above - it may exist but I don’t know of it.


#16

My own approach here is largely based on assumptions I’ve drawn based on watching how Juju behaves as I interact with it, and then referring to docs when it doesn’t do what I expect it to. Hardly the most efficient approach but that could be a good(polite) description of a lot of what I do :slight_smile:

I’d be happy to post my own interpretation of what I think the flags and state machine behavior do, if only for you lot to tell me how wrong I have got it - but perhaps it’ll make a useful jumping off point for your own correct documentation?