Watching containers with Portainer and Sysdig

[screen capture: csysdig running inside Portainer]

The question to be answered is how to manage Docker containers - not just how to get them running, but also how to poke inside them while they are running to see what they are doing and make sure that you can make sense of what is happening while you develop or run in production.

At Tectonic I was introduced to one tool that handles Docker and Kubernetes container introspection, Sysdig. From Twitter I also discovered Portainer, a nice system for container management. Having installed both Portainer and Sysdig, the interesting bit happens when you connect the together.

Portainer provides a very high level view of your container infrastructure, with an easy browser level access to see what containers you have loaded on the system, what’s running, and how to start or restart or pull new containers. Sysdig on the other hand has a very low level kernel’s eye view of the system, watching every system call and timing it and providing a system browser (with “csysdig”) that is reminiscent of faithful Unix tools like ‘htop’.

First, install each of them according to package instructions. As of this writing (December 2016) Portainer will run on Intel and ARM platforms, but Sysdig really wants to run on Intel only.

Portainer install instructions will want you to pull the latest version (v1.11) and then set a password. The vision for Portainer product direction is full role based access control, so expect developments here; just know that for now, if you want an admin password to persist, you’ll have to persist the data in the filesystem or in a Docker storage volume. As installs go, this is pretty easy.

Next, you’ll want to follow the Sysdig install instructions for installing Sysdig in a container. We’re monitoring CoreOS, which makes the effort directly supported; I haven’t done Ubuntu yet. Sysdig wants to have a kernel module installed, and the CoreOS install effort takes advantage of automated new kernel builds whenever CoreOS does a new release.

Once both systems are up, connect them together. You’ll want to connect to a running sysdig container through the console, via the navigation path dashboard / containers / sysdig / console. Launch a console window on your Sysdig container, and in that console run “csysdig -pc”. You’ll have a view on the innards of your containers.

Now the biggest missing piece is the amount of this narrative given over to install instructions, rather than use instructions. Portainer has some very nice ways to install a variety of containerized applications, but it doesn’t know how to launch Sysdig in a single click, nor is it reasonable to naively assume that it can. Even once you get the command line invocation right there may be a pesky requirement to get kernel headers all exactly correct. Still, if possible that would be the direction I’d give to both organizations - make it easier to embed Sysdig tools and functions inside a Portainer framework.