Agent introspection


#1

Sometimes software doesn’t do what you’d expect. Each of the agents that Juju runs has an internal worker for responding to introspection requests.

As the agents start up, a goroutine is started to listen on an abstract domain socket. The listener talks HTTP, and has a number of registered endpoints. The initial work was to expose the internal golang runtime debugging endpoints for getting access to the running goroutines, CPU profiles, and memory heap profiles. This was then extended to add additional endpoints for much more Juju specific information.

The Juju machine agent writes out a file to /etc/profile.d/juju-introspection.sh that defines a number of functions to easily get information out of the agent. These function names changed in Juju 2.3.9 and 2.4.2 to use underscores instead of dashes.