Simplifying the process of running local acceptance tests


#1

I’ve just proposed a PR to add a script to the core juju repository that gets us closer to having a “1 click” process for setting up a local environment for running acceptance tests.

Here is what using it looks like:

python create_juju_test_env.py /tmp/juju-123
warning 1: JUJU_HOME environment variable (/home/tsm/Work/src/github.com/juju/juju) is not set to /tmp/juju-123.
warning 2: JUJU_DATA environment variable (<unset>) is not set to /tmp/juju-123/data.
info: to run an acceptance test, here is the suggested command:
info: JUJU_HOME=/tmp/juju-123 JUJU_DATA=/tmp/juju-123/data ./assess_model_migration.py lxd $GOPATH/bin/juju /tmp/artifacts

As you can see, create_juju_test_env.py is quite noisy. It includes a few other checks, such as that $GOPATH and $JUJU_REPOSITORY are available.

Thoughts? If people are happy, I’ll update the README in juju/juju/acceptancetests to make use of it.


#2

Anything to help make running acceptance tests easier is great.
Do we need to log the messages about the env vars as warning? Can they be debug instead? Most people won’t have either of those set at all. Maybe we could have the script simply set those vars and actually run the test directly instead of asking the user to cut and paste?


#3

That’s the point. If they are set, and if they’re set correctly, then the warnings don’t appear.


#4

I don’t think that this is possible. Once a sub-process dies, its environment dies with it. I’m not sure if it’s possible to set an environment variable that propagates to the parent process, e.g. the shell where the script is invoked.


#5

The expected setup on people’s machine is not to have these env vars set. If we are adding a script to make running the tests easier, it should not print warnings in the most common, expected scenario. Otherwise it’s noise and makes people think their system is set up incorrectly when it’s not.


#6

This doesn’t make sense to me. You’re saying that you would rather not advise the user that their current environment won’t work for what they’re likely to do next, e.g. run an acceptance test?


#7

People’s current setup is geared to running juju. If we are adding a script to make running tests easier, that script should do what’s needed out of the box to run the specified test. It would need to create whatever staging dir(s) are needed to host the juju home and pass the necessary env vars to the test. And when the test has finished, people’s systems should remain as they were with no env vars messed with. That’s how we run the juju unit tests for example.