chroot
, namespaces
, and cgroups
.chroot
changes the file system root (/
) that a process can see. This allows a process to use any directory as if it were a file system root instead of the actual file system root.namespaces
group resources together (like network and process IDs) so that only processes within a namespace can see the resources of that namespace.cgroups
set CPU and memory limits for processes.Dockerfile
in the current working directory and tell Docker build it:docker build
command clearly show how Docker executes the build based on our Dockerfile.docker
CLI is the client that you use to send commands and data to the Docker daemon. The Docker daemon stores and runs containers. First, the docker build .
command tells Docker to use the local current working directory (.
) as the Docker context. The Docker context is sent to the Docker daemon.FROM python
step tells Docker to build the new container starting with the python container image from Docker Hub as the base.COPY . /public
copies all the files in the Docker context into the /public
directory on our container image. This includes all the files in our local current working directory that was sent over to the daemon as the Docker context.WORKDIR /public
sets the working directory for the container image we are building, so that when the container image is run, the current working directory for the process is /public
.WORKDIR
command is removed. Steps that do not change the container file system only modify the container configuration and are removed. The container configuration is metadata that describes how to run the container entrypoint process.ENTRYPOINT ...
sets the command to run when the container is run, which, in this case, runs an HTTP server that serves all the files in /public
. So, for example, if we had HTML files in our local current working directory, those would be served by this container.5550043a7340
, which can be used in subsequent docker
commands.localhost:8000
. If you had, say, an index.html
in your local current working directory, going to localhost:8000
would serve the contents of index.html
.RUN <command>
, which executes the <command>
using whichever shell is present in the container, creating a new container following the result of executing that command.RUN
s can usually be combined:FROM
instruction) builds a single image starting with the FROM
image as the base. These builds run in a purely linear format from start to finish and the results of all intermediate steps become part of the resulting image. However, since a container image should only contain what is necessary to run your application, much of the files from intermediate steps are junk that is not needed at run-time.maven
image as the base for an intermediate image called builder
.builder
image.mvn package
builds the fatjar for the project.openjdk
image that just has the JRE.builder
image..wh.
.TODO: Show some bash commands that can generate a layer
sha256:0b56ce23cf77e0a18c94b4ca8fe5fa58f4cca119f61ba4021cde2c9db56faaa4
.architecture
and os
fields define the platform this container is built to run on. The config
field defines the runtime configuration for the container, including any environment variables (Env
), the entrypoint (Entrypoint
), arguments to append to the entrypoint (Cmd
), ports to expose (ExposedPorts
), arbitrary annotations (Labels
), the directory to use as the current working directory (WorkingDir
), and the user to run the container process as (User
).TODO: Might want to talk about reasonings for burning runtime config into an image vs. defining in a runtime spec
TODO: Show example container configuration and running with Docker/Kubernetes and effective runtime configuration
rootfs
object and the history
array. Both contain metadata about the layers that make up the container file system.rootfs
contains a list of diff IDs. A diff ID for a layer is the digest (usually SHA256 hash) of the uncompressed tarball archive containing the files for that layer. Note that this is different from a layer's descriptor digest, which is a hash of the compressed archive. The rootfs
defines a list of these diff IDs in the order in which the layers belong in the container overlay file system, from first to last. Note that these layers must match those defined in the manifest.docker load
formatdocker load
can load a container image stored as a tarball archive into the Docker local repository store. This tarball archive includes the compressed tarball archives for all the layers, the container configuration JSON file, and the manifest JSON file (must be named manifest.json
). Here’s an example of the manifest JSON file:manifest.json
and reads it. The manifest tells Docker to load the container configuration from the config.json
file and load the layers from layer1.tar.gz
, layer2.tar.gz
, and layer3.tar.gz
, in that order. Note that this order must match the order of the layers defined in the container configuration under rootfs
. The RepoTags
here tells Docker to "tag" the image with the name myimage
.docker save
formatdocker save
can also save images in a legacy Docker tarball archive format that docker load
also supports. In this legacy format, each layer would be stored in its own directory named with its SHA256 hash (digest of compressed layer tarball archive). Each of these directories contains a json
file with a legacy container configuration (we won’t go into the details of this), a VERSION
file, and a layer.tar
that is the uncompressed tarball archive of the layer contents.schemaVersion
- 2
in this casemediaType
- application/vnd.docker.distribution.manifest.v2+json
config
- descriptor of container configuration bloblayers
- list of descriptors of layer blobs, in the same order as the rootfs
of the container configurationmediaType
- application/vnd.docker.container.image.v1+json
for a container configuration or application/vnd.docker.image.rootfs.diff.tar.gzip
for a layersize
- the size of the blob, in bytesdigest
- the digest of the contentbusybox
image is composed of a container configuration stored as a blob with digest sha256:3a093384…
and a single layer blob with digest sha256:57c14dd6…
. Note that manifests simply contain references. The actual blob content is stored elsewhere and would need to be fetched and provided to a container run-time when running the image as a container.rootfs
of the container configuration is the digest of the same archive but uncompressed. Otherwise, container registries may give you a cryptic MANIFEST_INVALID
error.mediaType
- must be set to application/vnd.oci.image.manifest.v1+json
config.mediaType
- must be set to application/vnd.oci.image.config.v1+json
layers
must have mediaType
be either application/vnd.oci.image.layer.v1.tar+gzip
or application/vnd.oci.image.layer.v1.tar
.application/vnd.oci.image.layer.v1.tar
mediaType
.TODO
registry.hub.docker.com
).library/
(for example, busybox
refers to library/busybox
).latest
.openjdk:8-jre-alpine
openjdk
. Since the repository is only a single level and the registry is Docker Hub, the repository is actually resolved as library/openjdk
. The tag component is specified after a colon. In this example, the tag is 8-jre-alpine
. Although this tag is arbitrarily-defined, the maintainers of this repository chose to convey some information about the image through this tag. Here, this tag says that the specific image contains OpenJDK 8 with just the JRE (Java Runtime Environment) and Alpine (a tiny Linux distribution). Some other tags in the repository refer to the same image but contain more specific information, like 8u191-jre-alpine3.9
. Other tags may be less specific, like jre
, which, at the time of writing, refers to an OpenJDK 11 image with the JRE and Debian Stretch. This tag will most likely move to refer to newer OpenJDK versions as they become GA. The latest tag also refers to the latest stable, default image the maintainers believes users will find useful in most cases and will move the tag as new versions become stable. There are some general tips and best practices for managing tags for a repository. TODO: Maybe explain some tips?
gcr.io/my-gcp-project/my-awesome-app/message-service:v1.2.3
gcr.io
, which is the server URL for Google Container Registry. The repository is my-gcp-project/my-awesome-app/message-service
, which in GCR, means that the repository is on the my-gcp-project
Google Cloud Platform project and under a my-awesome-app/message-service
Google Cloud Storage subdirectory. The tag is v1.2.3
, which is used to identify the version of the message-service
this container image contains.gcr.io/distroless/[email protected]:0430beea631408faa266e5d98d9c5dcc3c5f02c3ebc27a65d72bd281e3576097
.docker
CLI tool, "tag" also refers to a label given to an image on the Docker daemon. For example, you might build an image called myimage
with:4e905a76595b
. This image ID is a shortened SHA256 digest of the built container configuration. Docker then "tags" the 4e905a76595b
with a label myimage
, which can be used to refer to the same image with future docker
commands rather than using 4e905a76595b
all the time. However, when you want to build and push an image to a registry, you would "tag" the image with the full image reference:gcr.io/my-gcp-project/myimage:v1
to be used in later docker commands, but gcr.io/my-gcp-project/myimage:v1
is actually an image reference with its tag as v1
.openjdk
, the endpoint would be https://registry.hub.docker.com/v2/library/openjdk/manifests/latest
.Authorization
(explained in the Token Authentication section) and Accept
. For example, if you want to only accept and parse manifests for Docker Image Format V2 Schema 2, you would want to set Accept: application/vnd.docker.distribution.manifest.v2+json
.application/vnd.docker.distribution.manifest.list.v2+json
. These manifests lists contain specific manifests for different architectures and operating systems. For example, if you are looking specifically for a manifest for the amd64/Linux platform, you would parse the manifest list for the corresponding manifest for that platform and then pull that manifest using its specific digest. See an example of a manifest list.202 Accepted
response should contain a Location
header with the URL to send the blob content to. There are a few ways to send the blob content, but the recommended way is to send the content in a single request. The content can also be sent in chunks if you wish to implement resumable uploads. Send the blob content to the Location
via:202 Accepted
. Once a blob is uploaded, the blob can be committed with its digest:201 Created
.200 OK
.201 Created
. If the blob mount fails (blob doesn’t exist or cross-repository blob mount not supported), the response will be a normal blob upload initialization response with a Location
to send the blob content to.Authorization
to send with this request must have permissions to pull from the source repository in addition to permissions to push to the target repository. See the Token Authentication section below for details.Content-Type
to the correct manifest media type. For Docker V2.2, the media type would be application/vnd.docker.distribution.manifest.v2+json
and for OCI, the media type would be application/vnd.oci.image.manifest.v1+json
.Authorization
header to be authenticated. Otherwise, a 401 Unauthorized
or 403 Forbidden
response would be received.Authorization
header containing the credentials for access to the repository:coollog
account was donthackme123
, the Authorization
header would be Authorization: Basic Y29vbGxvZzpkb250aGFja21lMTIz
, where Y29vbGxvZzpkb250aGFja21lMTIz
is the Base64 encoding of coollog:donthackme123
.token
or access_token
(OAuth2) that is the authenticated Bearer token. The JSON also may also contain an expires_in
and issued_at
field for knowing when the token expires. These tokens should be cached and applied to any subsequent requests (until expired) to skip steps 1 and 2.scope
s can be included. For example, for a cross-repository blob mount with otheruser/baseimage
as the source repository, the request may be:Authorization
for both scopes though.docker login
). This configuration can be found at the .docker/config.json
file located in the user’s home directory. This configuration can store both raw credentials and references to credential helpers. Credential helpers are CLI tools that can be used to fetch credential.auths
field that is a map with registry URLs as keys. For example:credHelpers
field that maps from registry URLs to credential helper names. For example:credsStore
field. For example:gcr.io
:get
command, passing in the server URL as the input. For example, to fetch credentials from the credential helper named gcr
for the gcr.io
registry, call:Username
and Secret
field which can be used as the username and password, respectively, for authenticating a token.Username
returned is <token>
, the Secret
is actually an identity token that should be used as a refresh token in obtaining an OAuth2 access token. For example, Azure's credential helper returns a JSON that looks like:Secret
should be used as the refresh token for an OAuth2 access token request:blobs/
and selectors/
. The blobs/
directory contains files for each blob named by their digest. The selectors/
directory contains selector files whose names are the selectors themselves and contents are the digests of the blobs they point to. Both blobs and selectors should be immutable and unique.65de3b72…
selector file contains the 36a2b740…
digest. See the proposal for the original Jib cache design for more details.00
through ff
, with each file being placed into the bin that matches the first two characters in its file name.busybox
image from Docker Hub using some bash
commands.busybox
image. We curl
the registry API root to get the WWW-Authenticate
challenge:busybox
. Since busybox
is a public image, we don't need to include any Authorization
.busybox
:manifest.json
for docker load
:tar
up the entire image and load it into Docker:busybox
image we just pulled:bash
. We'll build a simple image that has a shell script that prints "Hello World"
on top of the busybox
image."Hello World"
shell script and archive it into its own layer tarball:<username>/hello
. First, we need to push our layers. The first layer comes from busybox
, so we can actually just mount that into our new image:busybox
layer into our <username>/hello
repository.401 Unauthorized
.hello.sh
layer:Location
received:Location
received:TODO
Separate into fine layers Speed up iterative build cycle - separate out parts that change more often into another layer Minimize change stream everything through sinks/sources