Trouble with aws-integrator


#1

I’m trying to deploy a charm mentioned in post Getting Started that involves the aws-integrator charm:

juju bootstrap --credential juju-k8s --config charmstore-url=https://api.staging.jujucharms.com/charmstore aws aws-k8s
juju deploy cs:~johnsca/kube-core-aws
juju trust aws-integrator

I end up with a failure.

Credential ‘juju-k8s’ is associated with the AWS ‘AdministratorAccess’ policy and the custom policy I created specifically for ‘aws-integrator’.

Any ideas?


#2

The credential access rights look correct to me. What’s the content of the hook error as seen in the debug logs? That will help the aws integrator charm folks diagnose the issue.


#3

http://paste.ubuntu.com/p/W5bmscFKXh/


#4

the first thing I would do is use the latest aws-integrator charm. Current revision is 7, and you seem to have 4

juju upgrade charm should do it


#5

Doesn’t help. I guess the staging Charm Store, which I need, doesn’t have the newer revision.

juju upgrade-charm aws-integrator
ERROR already running latest charm "cs:~johnsca/aws-integrator-4"

#6

The JAAS team helpfully sync’d the ~containers space. Try using the charm straight from there. It should be the latest version as of yesterday.


#7

I downloaded the bundle I was using and replaced the default aws-integrator charm with, as Rick suggested:

    charm: "cs:~containers/aws-integrator"

This helped, yet I still get a permission error:

http://paste.ubuntu.com/p/jJm2tgR7yr/

As a test I’ve granted to my Juju credential the IAM AdministratorAccess role. I also, probably futilely, added all IAM “actions” within the “EC2 service”. I’m not sure how to give more than that.


#8

@cory_fu Any ideas on this?


#9

That error is coming from this query:

    subnet_id = _aws('ec2', 'describe-instances',
                     '--instance-ids', instance_id,
                     '--region', region,
                     '--query', 'Reservations[*]'
                                '.Instances[*]'
                                '.SubnetId[] | [0]')

I’ve never seen that fail before and I’m not sure what could cause it. Possibly the credentials provided to the integrator charm lacking the EC2 DescribeInstances permission, or maybe some sort of issue with the subnet being attached? Maybe you can try manually running that query and provide the results?


#10

I have given my user all the permissions I can think of (as stated earlier). This is the user whose credentials Juju is using to bootstrap.

As for manually querying, I presume I need to install an AWS CLI client. I’ll need to set that up.


#11

You could also just ssh into the aws-integrator unit and run the query from there. If you do the following, the credential will be already configured:

sudo su
export HOME=/root

#12

Also, make sure to leave off the final | [0] bit of the query to see all of the subnet info, and possibly all of the .SubnetId[] | [0] bit to see all instance info.


#13

I’m afraid this is going to betray my cluelessness.

I tried running the function as is to begin and got an error:

# cd /var/lib/juju/agents/unit-aws-integrator-0/charm/lib/charms/layer
# python3 -c 'from aws import *; print tag_instance_subnet(instance_id, region, tags)'
File "<string>", line 1
from aws import *; print tag_instance_subnet(instance_id, region, tags)
                                           ^
SyntaxError: invalid syntax

#14

I made a change to how I called the function and got at least some semblance of next steps:

python3 -c 'from aws import *; print(tag_instance_subnet(instance_id, region, tags))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/var/lib/juju/agents/unit-aws-integrator-0/charm/lib/charms/layer/aws.py", line 14, in <module>
    from charmhelpers.core import hookenv
ModuleNotFoundError: No module named 'charmhelpers'

#15

The ModuleNotFound error seems like you need to ensure that the charm’s venv is activated. You could try using charm-env python3 ... instead. However, even if that runs, I suspect that it would just give you the same error that’s in the Juju log. Instead, I think you need to run a modified form of the query directly, to see what values the query is coming back with for the subnets.

Would you mind pasting the results of the following, filling in the $REGION and $INSTANCE_ID from the output of juju status:

juju run --unit aws-integrator/0 -- aws ec2 describe-instances --profile juju --output json --instance-ids $INSTANCE_ID --region $REGION --query 'Reservations[*].Instances[*].SubnetId[]'

#16

I’d first like to re-explain my environment since my initial post. Since we spoke (on IRC) about bundle ‘kube-core-aws’ I have since move to just using bundle ‘canonical-kubernetes’. I then installed ‘aws-integrator’ separately (juju deploy cs:~containers/aws-integrator).

After having run juju trust aws-integrator the final result is here.

Pre-pending charm-env gave me a different error:

charm-env python3 -c 'from aws import *; print(tag_instance_subnet(instance_id, region, tags))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/var/lib/juju/agents/unit-aws-integrator-0/charm/lib/charms/layer/aws.py", line 22, in <module>
    MODEL_UUID = os.environ['JUJU_MODEL_UUID']
  File "/var/lib/juju/agents/unit-aws-integrator-0/.venv/lib/python3.6/os.py", line 669, in __getitem__
    raise KeyError(key) from None
KeyError: 'JUJU_MODEL_UUID'

Details here.

Now the result of the juju run command:

juju run --unit aws-integrator/0 -- aws ec2 describe-instances --profile juju --output json --instance-ids i-09128b6fbacebdb32 --region us-east-1 --query 'Reservations[*].Instances[*].SubnetId[]'
/tmp/juju-exec418336576/script.sh: line 1: aws: command not found