GDI Logo

Intro to Docker

Class 2

Welcome!

Girl Develop It is here to provide affordable and accessible programs to learn software through mentorship and hands-on instruction.

Some "rules"

  • We are here for you!
  • Every question is important
  • Help each other
  • Have fun

What we will cover today

  • Building docker images
  • Copying files during a build
  • Advanced Dockerfiles
  • Naming & inspecting containers
  • Resources

Building Docker Images

Building Images Interactively

As we have seen, the images on the Docker Hub are sometimes very basic.

How do we want to construct our own images?

As an example, we will build an image that has figlet.

First, we will do it manually with docker commit.

Then later, we will use a Dockerfile and docker build.

Building from a base

Our base will be the ubuntu image.

Create a new container and make some changes

Start an Ubuntu container:

$ docker run -it ubuntu
root@<yourContainerId>:#/

Run the command apt-get update to refresh the list of packages available to install.

Then run the command apt-get install figlet to install the program we are interested in.

root@:#/ apt-get update && apt-get install figlet
.... OUTPUT OF APT-GET COMMANDS ....

Inspect the changes

Type exit at the container prompt to leave the interactive session.

Now let's run docker diff to see the difference between the base image and our container.

$ docker diff <yourContainerId>
C /root
A /root/.bash_history
C /tmp
C /usr
C /usr/bin
A /usr/bin/figlet
...

Docker tracks filesystem changes

As we know:

  • An image is read-only.
  • When we make changes, they happen in a copy of the image.
  • Docker can show the difference between the image, and its copy.
  • For performance, Docker uses copy-on-write systems. (i.e. starting a container based on a big image doesn't incur a huge copy.)

Commit and run your image

The docker commit command will create a new layer with those changes, and a new image using this new layer.

$ docker commit <yourContainerId>
<newImageId>

The output of the docker commit command will be the ID for your newly created image.

We can run this image:

$ docker run -it <newImageId>
root@fcfb62f0bfde:/# figlet hello
| |__   ___| | | ___
| '_ \ / _ \ | |/ _ \
| | | |  __/ | | (_) |
|_| |_|\___|_|_|\___/

Tagging images

Referring to an image by its ID is not convenient. Let's tag it instead.

We can use the tag command:

$ docker tag <newImageId> figlet

But we can also specify the tag as an extra argument to commit:

$ docker commit <containerId> figlet

And then run it using its tag:

$ docker run -it figlet

What's next?

Manual process = bad.

Automated process = good.

Next we will learn how to automate the build process by writing a Dockerfile.

Building Images with a Dockerfile

  • A Dockerfile is a build recipe for a Docker image.
  • It contains a series of instructions telling Docker how an image is constructed.
  • The docker build command builds an image from a Dockerfile.

Writing our first Dockerfile

Our Dockerfile must be in a new, empty directory.

  1. Create a directory to hold our Dockerfile.
    $ mkdir myimage
  2. Create a Dockerfile inside this directory.
    $ cd myimage
$ vim Dockerfile

Of course, you can use any other editor of your choice.

Type this into our Dockerfile

FROM ubuntu
RUN apt-get update
RUN apt-get install figlet
  • FROM indicates the base image for our build.
  • Each RUN line will be executed by Docker during the build.
  • Our RUN commands must be non-interactive. (No input can be provided to Docker during the build.)
  • In many cases, we will add the -y flag to apt-get.

Build it!

Save our file, then execute:

$ docker build -t figlet .
  • -t indicates the tag to apply to the image.
  • . indicates the location of the build context. (We will talk more about the build context later; but to keep things simple: this is the directory where our Dockerfile is located.)

What happens when we build the image?

The output of docker build looks like this:

$ docker build -t figlet .
Sending build context to Docker daemon 2.048 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu
 ---> e54ca5efa2e9
Step 1 : RUN apt-get update
 ---> Running in 840cb3533193
 ---> 7257c37726a1
Removing intermediate container 840cb3533193
Step 2 : RUN apt-get install figlet
 ---> Running in 2b44df762a2f
 ---> f9e8f1642759
Removing intermediate container 2b44df762a2f
Successfully built f9e8f1642759
  • The output of the RUN commands has been omitted.
  • Let's explain what this output means.

Sending the build context to Docker

Sending build context to Docker daemon 2.048 kB
  • The build context is the . directory given to docker build.
  • It is sent (as an archive) by the Docker client to the Docker daemon.
  • This allows to use a remote machine to build using local files.
  • Be careful (or patient) if that directory is big and your link is slow.

Executing each step

Step 1 : RUN apt-get update
 ---> Running in 840cb3533193
(...output of the RUN command...)
 ---> 7257c37726a1
Removing intermediate container 840cb3533193
  • A container (840cb3533193) is created from the base image.
  • The RUN command is executed in this container.
  • The container is committed into an image (7257c37726a1).
  • The build container (840cb3533193) is removed.
  • The output of this step will be the base image for the next one.

The caching system

If you run the same build again, it will be instantaneous.

Why?

  • After each build step, Docker takes a snapshot of the resulting image.
  • Before executing a step, Docker checks if it has already built the same sequence.
  • Docker uses the exact strings defined in your Dockerfile, so:
    • RUN apt-get install figlet cowsay is different from RUN apt-get install cowsay figlet
    • RUN apt-get update is not re-executed when the mirrors are updated

You can force a rebuild with docker build --no-cache

Running the image

The resulting image is not different from the one produced manually.

$ docker run -ti figlet
root@91f3c974c9a1:/# figlet hello
| |__   ___| | | ___
| '_ \ / _ \ | |/ _ \
| | | |  __/ | | (_) |
|_| |_|\___|_|_|\___/

Sweet is the taste of success!

Using image and viewing history

The history command lists all the layers composing an image.

For each layer, it shows its creation time, size, and creation command.

When an image was built with a Dockerfile, each layer corresponds to a line of the Dockerfile.

$ docker history figlet
IMAGE         CREATED            CREATED BY                     SIZE
f9e8f1642759  About an hour ago  /bin/sh -c apt-get install fi  1.627 MB
7257c37726a1  About an hour ago  /bin/sh -c apt-get update      21.58 MB
07c86167cdc4  4 days ago         /bin/sh -c #(nop) CMD ["/bin   0 B
<missing>     4 days ago         /bin/sh -c echo '#!/bin/sh'    194.5 kB

Introducing JSON Syntax

Most Dockerfile arguments can be passed in two forms:

  • plain string: RUN apt-get install figlet
  • JSON list: RUN ["apt-get", "install", "figlet"]

    • Let's change our Dockerfile as follows!

    FROM ubuntu
RUN apt-get update
RUN ["apt-get", "install", "figlet"]

    Then build the new Dockerfile.

    $ docker build -t figlet .

JSON syntax vs string syntax

Compare the new history:

$ docker history figlet
  • JSON syntax specifies an exact command to execute.
  • String syntax specifies a command to be wrapped within /bin/sh -c "...".

Develop It!

Spend this time working on Docker Labs Exercise 2.0: Webapps with Docker

Copying the files during the build

So far, we have installed things in our container images by downloading packages.

We can also copy files from the build context to the container that we are building.

Remember: the build context is the directory containing the Dockerfile.

Next, we will learn a new Dockerfile keyword: COPY.

Build some C code

We want to build a container that compiles a basic "Hello world" program in C.

Here is the program, hello.c:

int main () {
puts("Hello, world!");
return 0; }

Let's create a new directory, and put this file in there.

Then we will write the Dockerfile.

The Dockerfile

On Debian and Ubuntu, the package build-essential will get us a compiler.

When installing it, don't forget to specify the -y flag, otherwise the build will fail (since the build cannot be interactive).

Then we will use COPY to place the source file into the container.

FROM ubuntu
RUN apt-get update
RUN apt-get install -y build-essential
COPY hello.c /
RUN make hello
CMD /hello

Create this Dockerfile.

Testing our C program

  • Create hello.c and Dockerfile in the same directory.
  • Run docker build -t hello . in this directory.
  • Run docker run hello, you should see Hello, world!.

Success!

COPY and the build cache

  • Run the build again.
  • Now, modify hello.c and run the build again.
  • Docker can cache steps involving COPY.
  • Those steps will not be executed again if the files haven't been changed.

Details

  • You can COPY whole directories recursively.
  • Older Dockerfiles also have the ADD instruction. It is similar but can automatically extract archives.
  • If we really wanted to compile C code in a compiler, we would:
    • Place it in a different directory, with the WORKDIR instruction.
    • Even better, use the gcc official image.

Advanced Dockerfiles

Usage Summary

  • Dockerfile instructions are executed in order.
  • Each instruction creates a new layer in the image.
  • Instructions are cached. If no changes are detected then the instruction is skipped and the cached layer used.
  • The FROM instruction MUST be the first non-comment instruction.
  • Lines starting with # are treated as comments.
  • You can only have one CMD and one ENTRYPOINT instruction in a Dockerfile.

The FROM instruction

  • Specifies the source image to build this image.
  • Must be the first instruction in the Dockerfile, except for comments.

The FROM instruction

Can specify a base image:

FROM ubuntu

An image tagged with a specific version:

FROM ubuntu:12.04

A user image:

FROM training/sinatra

Or self-hosted image:

FROM localhost:5000/funtoo

More about FROM

The FROM instruction can be specified more than once to build multiple images.

FROM ubuntu:14.04 .. .
FROM fedora:20 .. .
  • Each FROM instruction marks the beginning of the build of a new image. The -t command-line parameter will only apply to the last image.
  • If the build fails, existing tags are left unchanged.
  • An optional version tag can be added after the name of the image. E.g.: ubuntu:14.04.

A use case for multiple FROM lines

Integrate CI and unit tests in the build system

FROM <baseimage>
RUN <install dependencies>
COPY <code>
RUN <build code>
RUN <install test dependencies>
COPY <test data sets and fixtures>
RUN <unit tests>
FROM <baseimage>
RUN <install dependencies>
COPY <vcode>
RUN vbuild code>
CMD, EXPOSE ...
  • The build fails as soon as an instructions fails
  • If RUN fails, the build doesn't produce an image
  • If it succeeds, it produces a clean image (without test libraries and data)

The MAINTAINER instruction

The MAINTAINER instruction tells you who wrote the Dockerfile.

MAINTAINER Docker Education Team <education@docker.com>

It's optional but recommended.

The RUN instruction

The RUN instruction can be specified in two ways.

With shell wrapping, which runs the specified command inside a shell, with /bin/sh -c:

RUN apt-get update

Or using the exec method, which avoids shell string expansion, and allows execution in images that don't have /bin/sh:

RUN [ "apt-get", "update" ]

More about the RUN instruction

RUN will do the following:

  • Execute a command.
  • Record changes made to the filesystem.
  • Work great to install libraries, packages, and various files.

RUN will NOT do the following:

  • Record state of processes.
  • Automatically start daemons.

If you want to start something automatically when the container runs, you should use CMD and/or ENTRYPOINT.

Collapsing layers

It is possible to execute multiple commands in a single step:

RUN apt-get update && apt-get install -y wget && apt-get clean

It is also possible to break a command onto multiple lines:

It is possible to execute multiple commands in a single step:

RUN apt-get update \
&& apt-get install -y wget \
&& apt-get clean

The EXPOSE instruction

The EXPOSE instruction tells Docker what ports are to be published in this image.

EXPOSE 8080
EXPOSE 80 443
EXPOSE 53/tcp 53/udp
  • All ports are private by default.
  • The Dockerfile doesn't control if a port is publicly available.
  • When you docker run -p ..., that port becomes public. (Even if it was not declared with EXPOSE.)
  • When you docker run -P ... (without port number), all ports declared with EXPOSE become public.

The EXPOSE instruction

A public port is reachable from other containers and from outside the host.

A private port is not reachable from outside.

The COPY instruction

The COPY instruction adds files and content from your host into the image.

COPY . /src

This will add the contents of the build context (the directory passed as an argument to docker build) to the directory /src in the container.

Note: you can only reference files and directories inside the build context. Absolute paths are taken as being anchored to the build context, so the two following lines are equivalent:

COPY . /src
COPY / /src

The COPY instruction

Attempts to use .. to get out of the build context will be detected and blocked with Docker, and the build will fail.

Otherwise, a Dockerfile could succeed on host A, but fail on host B.

ADD

ADD works almost like COPY, but has a few extra features.

ADD can get remote files:

ADD http://www.example.com/webapp.jar /opt/

This would download the webapp.jar file and place it in the /opt directory.

ADD will automatically unpack zip files and tar archives:

ADD ./assets.zip /var/www/htdocs/assets/

This would unpack assets.zip into /var/www/htdocs/assets. However, ADD will not automatically unpack remote archives.

ADD, COPY and the build cache

  • For most Dockerfiles instructions, Docker only checks if the line in the Dockerfile has changed.
  • For ADD and COPY, Docker also checks if the files to be added to the container have been changed.
  • ADD always need to download the remote file before it can check if it has been changed. (It cannot use, e.g., ETags or If-Modified-Since headers.)

More Advanced Dockerfile Instructions

  • VOLUME: tells Docker that a specific directory should be a volume.
  • WORKDIR: sets the working directory for subsequent instructions.
  • ENV: specifies environment variables that should be set in any container launched from the image.
  • USER: sets the user name or UID to use when running the image.
  • CMD: a default command run when a container is launched from the image.
  • ENTRYPOINT: like the CMD instruction, but arguments given on the command line are appended to the entry point.

More Advanced Dockerfile Instructions

  • ONBUILD: lets you stash instructions that will be executed when this image is used as a base for another one.
  • LABEL: adds arbitrary metadata to the image.
  • ARG: defines build-time variables (optional or mandatory).
  • STOPSIGNAL: sets the signal for docker stop (TERM by default).
  • HEALTHCHECK: defines a command assessing the status of the container.
  • SHELL: sets the default program to use for string-syntax RUN, CMD, etc.

Naming and Inspecting Containers

So far, we have referenced containers with their ID.

We have copy-pasted the ID, or used a shortened prefix.

But each container can also be referenced by its name.

If a container is named prod-db, I can do:

$ docker logs prod-db
$ docker stop prod-db
etc.

Default names

When we create a container, if we don't give a specific name, Docker will pick one for us.

It will be the concatenation of:

  • A mood (furious, goofy, suspicious, boring...)
  • The name of a famous inventor (tesla, darwin, wozniak...)


Examples: happy_curie, clever_hopper, jovial_lovelace

Specifying a name

You can set the name of the container when you create it.

$ docker run --name ticktock jpetazzo/clock

If you specify a name that already exists, Docker will refuse to create the container.

This lets us enforce unicity of a given resource.

Renaming containers

  • You can rename containers with docker rename.
  • This allows you to "free up" a name without destroying the associated container.

Inspecting a container

$ docker inspect <containerID>
[{
"AppArmorProfile": "",
"Args": [],
"Config": {
    "AttachStderr": true,
    "AttachStdin": false,
    "AttachStdout": true,
    "Cmd": [
    "bash"
    ],
    "CpuShares": 0,
...

There are multiple ways to consume that information.

Parsing JSON with the Shell

  • You could grep and cut or awk the output of docker inspect.
  • Please, don't.
  • It's painful.
  • If you really must parse JSON from the Shell, use JQ! (It's great.)
$ docker inspect <containerID> | jq .

We will see a better solution which doesn't require extra tools.

Using --format

You can specify a format string, which will be parsed by Go's text/template package. 

$ docker inspect --format '{{ json .Created }}' 
"2015-02-24T07:21:11.712240394Z"
  • The generic syntax is to wrap the expression with double curly braces.
  • The expression starts with a dot representing the JSON object.
  • Then each field or member can be accessed in dotted notation syntax.
  • The optional json keyword asks for valid JSON output. (e.g. here it adds the surrounding double-quotes.)

Resources

Resources

Questions?