By Nicholas Clements, Director of Engineering at CohesiveFT
One of the areas for which CohesiveFT is known is our technical support, with everyone pitching in. We provide such a high quality of support for our products that customers often ask about support and best practices for other products and services. One such event occurred recently when a customer was asking about best practices in the secure provisioning and configuration of Docker images and containers. Since we support the use of containers in version 3.5 of VNS3 Manager we were more than happy to talk.
The customer wanted to know how to create and customize Docker containers in a safe and secure manner. In general terms they wanted to have each container be individual with keys and configurations that would be applicable only to the group/user accessing the container. This would allow them to control who sees what and to easily add or revoke privileges. They wanted to distribute images and Dockerfiles without make them available to the world at large, they wanted to keep the generated credentials in a controlled, private repository.
And they wanted to do this without doing a ton of development work.
There are many ways to inject data into an image or a container. One approach used by many Docker aficionados is to keep the image itself suitable for public consumption and then pass sensitive information at container start time, either by environment variables or by mounting an external location into the container. Both the customer and CohesiveFT view that this approach compromises the “sandbox” aspect of Docker, although it is certainly a useful method to keep in the mental toolbox as Docker evolves.
|Image credit: Lifehacker|
This approach is not feasible with VNS3 -- the customer has no direct access to the VNS3 filesystem and we don’t (currently) support the passing of environment variables through our API or GUI. (We’re working on it. Promise. Maybe in version 3.7.)
Currently, customers can upload images directly or have the VNS3 appliance build them into the network platform. Customers can either start from scratch using Dockerfiles and public repositories or they can upload their own images.
We ran through a few scenarios with the customer. Their view was that there were two immediate options available with minimal risk:
Running a configuration management client inside the container -- but on reflection it was decided that this just pushed the issue further down the road, since the same problems would need to be solved at a later date.
Building images with predefined SSH keys, running sshd in the container, and connecting after the fact to deliver the individual keys and config files etc -- Issues here include timing, plus the need to figure out a method for pre-generating and storing all of the keys (or using a common key which, it was agreed, is a bad idea).
Even though Docker 1.0 is now out, Docker is still evolving rapidly as are the accepted best practices. In order to protect our customers as much as possible we decided to approach things from a conservative perspective.
The goals were:
- Creation of a standard image in a secure manner, with as little exposure as possible
- Customization of that image on a per-VNS3 instance, per-container possible, securely
- Secure storage of modifications
- Minimal development
We started with a secure but accessible way for storing objects, image files, Dockerfiles, configuration and supporting files. This was simple and the final implementation choice was left to the customer. Our suggestion was to use an internal, behind-the-firewall web server and have it join the VNS3 topology. An alternative was to use S3 and signed URLs with a long expiration.
Next we examined the choice between the use of a Dockerfile or a pre-built image. Since the customer was looking at adding containers to a preexisting topology we recommended the latter. It would have a lesser effect on the network. The result is the same - when we use images we use Dockerfiles to build them and then export the resulting container.
Finally, per-manager customization. One of the features that we implemented in VNS3 3.5 is the ability to rebuild an image based on a locally stored one. This approach uses a Dockerfile but since the base image being used is local the impact on the manager is slight. And if the commands used in the Dockerfile are minimal then the overall rebuild is quick.
The initial proof-of-concept used the VNS3 GUI to import a base image (unimaginatively called “base_image”) with an unknown root password and personalize it through an external Dockerfile archive.
RUN mkdir -p /root/.ssh
RUN echo "[PUBLIC KEY HERE]" >> /root/.ssh/authorized_keys
RUN chmod 400 /root/.ssh/authorized_keys
It took a further couple of hours to put together a simple bash script that made use of the VNS3 API and pulled the public key from a private key server as a proof-of-concept.
So with a couple of lines of code it’s possible to securely create -- and, more importantly, securely destroy -- personalized Docker containers within VNS3. The customer didn’t need to spend a lot of time or money on an elegant solution to their problem. Security and control.