Upgrading a charm

See also: Juju OLM | juju refresh

A charm can be changed at runtime with the juju refresh command. By default, it changes to the latest available version of the same charm; if --revision is specified, it changes to that revision of the same charm; and if --switch is specified it changes to any arbitrary charm, inferred from the argument in the same way as in juju deploy.

For a charm to replace another, though, there is a minimum standard of compatibility, which applies regardless of the particular change. That is:

  • A subordinate charm must be replaced by a subordinate charm, and a principal charm must be replaced by a principal charm.
  • Every runtime relation used by the application must exist in both charms.
  • Charm relations that are defined, but not in use at runtime, may be removed freely.

No other factor is used in determining compatibility: configuration settings in particular are converted completely naively, such that any settings from the original charm that share a name and type are preserved; any incompatible settings are removed; and any new settings take defaults as though freshly deployed. Settings may be added or updated by specifying the --config flag, pointing to a YAML-encoded application config file (see Juju OLM | How to configure an application for general information on configuring a charm). It is important to read the documentation for the charm’s new version, not only to learn about new settings but especially to find out about any new settings that require values.

When an application has been upgraded but a particular unit has not, the unit will continue to see the configuration settings from before conversion; these settings will not be affected by subsequent changes to the application’s settings.

Forced charm upgrades

Juju defines the upgrade-charm hook for resolving differences between versions of the same charm. No notice is given of charm upgrades; a charm upgrade may run at any time the unit is started, and the only opportunity for resolution that exists occurs after the change has taken place.

This is quite a tight restriction, but nonetheless valuable, so long as you can guarantee it will run. However, it’s important to understand that the upgrade- charm accepts a --force-units flag: a forced charm upgrade will upgrade even units that are currently in an error state, at the cost of skipping the upgrade-charm hook for those units.

This is useful for charm authors who want to push a new version of a failed hook (they can juju refresh --force-units and then resolved to run it immediately without otherwise disturbing the system); but it’s potentially dangerous if abused. We recommend that use of the feature be restricted to charm authors while developing their own charms, and that it’s not sensible to devote serious effort to recovering from inappropriately forced upgrades.

Charm upgrade errors

These will only occur as a result of conflicts between the contents of the charm directory written at runtime, and should never be seen by a user; users certainly cannot be expected to understand the structure of your charm well enough to solve the conflicts sanely.

When you’re writing a new version of a charm, you should always test upgrading it from (at least) the previous version, to ensure these errors don’t slip out into the wild.

You can completely avoid these errors by never writing to the charm directory; and you can also avoid them by rigorously delineating the parts of your charm directory that you write to at runtime, and ensuring you never add a file to the raw charm that could conflict with the runtime state.

If you’re writing your hooks in Python, you should be doubly aware of this: if you don’t configure Python to suppress bytecode caching, it will write .pyc files next to your Python files at runtime, and effectively prevent you from rearranging those directories in future. This is not an unreasonable burden to bear, but it’s important to know you’re taking it on.

If you encounter a charm upgrade error, you can run git status in the charm directory to see what the problem is, and use the knowledge thus gleaned to fix the charm and try to upgrade again.

Upgrade mechanics

Here is a summary of how charm upgrades work.

The agent (running on each unit of the application being upgraded) unpacks the new version of the charm to a staging directory next to the original charm. It then points itself to the newly unpacked charm and deletes the old one, after which the expected hooks are fired using the newly unpacked charm code.

The original charm continues to run until the new charm is successfully downloaded and unpacked. Only then does the original one stop running and the new one begin to run (by executing the install and upgrade-charm hooks).

To be clear, the logic of the upgrade itself is contained within the charm. Juju simply unpacks the new charm and fires the hooks.