Everything Docker!!

What is Docker, and what problem it solves?

Docker is a open-source tool that packages application + its dependencies into a container so that it runs the same way everywhere.

”Build once, run anywhere”

Before Docker, there is this problem “App works on my laptop, fails on server”. The Reason? Because when we have multiple ENVs(OS versions, Libraries, Runtime configs) it is quite difficult to replicate them. Even if we fix the dependencies version or mismatch issue, some applications are build to run on one specific OS or ENV. With this, its very difficult/slow to scale the application.

Docker solves it by using containers.

What are containers? And what is the difference b/w image? and container?

Containers are running instance of image. Container are very light weight, easy to build/destroy, can be deployed on cloud and they have their own OS and own configs. Container is ephemeral

In easy words we can say, container are lightweight isolated environment that runs application, dependencies, runtime, etc. They share the Host OS kernal and starts in seconds.

Images are readonly templates/blueprints that are used to create containers. Images are immutable. Images can be created by Dockerfile, containers. They are stored in some registry(DockerHub, ECR) and they can’t run by itself.

In easy words, we can say images are OS and containers are machines where this OS is installed. We can create multiple container off single image and these containers are not connected anyhow, meaning they are independent.

What is a Port in Docker? Why Ports Need to Be Exposed?

A port is a communication endpoint that allows traffic to reach to an application that is running inside the container.

The reason be needed to expose port is For example, inside the container a service is running at port 9000, when we try to access the same application through browser or localhost, it is not reachable. The reason is, we already know containers are isolated, so it can’t interact outside its container. It is accessible only inside container. So we need to map the host port(Host port can be any available port) with our container’s application port to access it.

Ports needs to be exposed because containers are isolated by default, and port mapping allows external traffic to reach the application inside the container.

Browser → Host Port → Container Port → Application

What is Dockerfile? How to write one? What are different commands used in Dockerfile?

A Dockerfile is text-file with no extension(Only Dockerfile) that contains step-by-step instructions to build a docker image. Dockerfile is used for automating image creation, version controlled image builds, and it ensures consistent environments.

Basic Dockerfile structure:

FROM base_image
WORKDIR /path_to_directory
COPY /path_to_files . //copy files from host to container
RUN install dependencies //executes commands at build-time
EXPOSE Port //Define container application port
CMD ["npm", "start"] //Default commands when container starts

# Build & run
docker build -t myapp .
docker run -it -p 3000:3000 myapp

A Dockerfile is a declarative script that defines how a Docker image is built using layered instructions

Dockerfile commands

# Defines baseimage OS/runtime. Each FROM instruction starts a new "stage"
# First instruction (except ARG)
FROM ubuntu

# build-time variable. Available only during build.
# Not present in final image.
# Can be overridden: --build-arg
ARG VERSION=1.0

# Runtime ENV var. Persists in container
# Used by app/runtime. Overrides ARG if same name
ENV APP_ENV=production

# Working directory inside image. Creates dir if not exists
# Replaces cd. Affects RUN, CMD, COPY
WORKDIR /app

# Copy from host to container. Predictable & explicit.
# Respects .dockerignore
COPY /file_source_path .

# Advanced copy. Extract tar and supports url
ADD app.tar.gz /app

# Executes commands at build-time. Creates new image layer.
# Used for dependency install. Chain commands to reduce layers
RUN apt-get update && apt-get install -y curl

# Document container port. Does NOT publish port to host
EXPOSE 3000

# Default startup command. can be overridden at runtime
# Only one CMD allowed. Used with ENTRYPOINT
CMD ["node", "app.js"]

# Fixed startup command. Harder to override
# Ensures container runs as intended. Often paired with CMD
ENTRYPOINT ["python", "app.py"]

# Run container as non-root. Security best practice
# Avoid root containers. Required in prod setups
USER sahil

# Mount point for persistent data. Data survives container removal
# Anonymous by default. Common for DBs
VOLUME /data

# Metadata. Image documentation. Used for automation & audits
# Replaces deprecated MAINTAINER
LABEL maintainer="sahil@devops.com"

# Change shell for RUN. Needed for bash features. Useful in complex scripts
SHELL ["/bin/bash", "-c"]

# Reports healthy/unhealthy. Used by orchestrators
# Improves self-healing. Container health monitoring
HEALTHCHECK CMD curl -f <http://localhost> || exit 1

Docker Image Layers

Each Dockerfile instruction is one immutable layers. Layers are cached and reusable. Containers add a writable layer on top

Writable Container Layer
------------------------
Image Layer (CMD)
Image Layer (COPY)
Image Layer (RUN)
Image Layer (FROM)
------------------------
Host Kernel

Why layers matters?

• Faster builds (cache reuse)
• Smaller downloads
• Efficient CI/CD
• Versioned changes

Example Dockerfile with optimised layers

FROM node:18-alpine
WORKDIR /app
# COPY . .
COPY package.json package-lock.json ./
# BAD: Creates 3 distinct layers, leaving the apt cache permanently in the image
# RUN apt-get update
# RUN apt-get install -y curl
# RUN rm -rf /var/lib/apt/lists/*

# Optimized: Creates 1 lean layer
RUN apt-get update && \\
    apt-get install -y curl && \\
    rm -rf /var/lib/apt/lists/*
COPY . .
CMD ["npm", "start"]

In this example, if we copy everything before, when a slightest change in code results into Dockerfile rebuild. With this optimised version, build time reduced significantly, as it is cached now. Meaning it gets rebuilds completely only when packages are added/changed. Rest installation steps are cached.

.dockerignore file

It is like .gitignore file that excludes files from build context. With this we get smaller images, faster builds, no secret leakage and cleaner layers

node_modules
.git
.env

.dockerignore prevents unnecessary or sensitive files from being sent to Docker daemon during build.

Multi-Stage Dockerfiles

A multi-stage build allows you to use multiple FROM statements in a single Dockerfile. The magic happens when you tell Docker to copy specific files from a previous stage into your current stage, leaving all the bloated build tools behind.

Example Dockerfile

# STAGE 1: The Builder
# We name this stage "builder" so we can reference it later
FROM node:18-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm install
COPY . .
RUN npm run build
# At this point, the compiled files are sitting in /app/build

# STAGE 2: The Production Environment
# We start completely fresh with a tiny Nginx web server image
FROM nginx:alpine
# Copy ONLY the compiled files from the "builder" stage.
# We leave Node.js, node_modules, and the source code behind!
COPY --from=builder /app/build /usr/share/nginx/html
# Expose port and start Nginx
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
# Build stage
FROM node:18 AS builder
WORKDIR /app
COPY . .
RUN npm install && npm run build
# Runtime stage
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist .
CMD ["node", "app.js"]

Build commands:

# Step 1: Build the Image
docker build -t my-fast-app .
# Don't forget the . at the end!
# That tells Docker to look in the current directory for the Dockerfile.

# Step 2: Run the Container
docker run -d -p 8080:80 my-fast-app
# -d: Runs the container in the background (detached mode).
# -p 8080:80: Maps port 8080 on your laptop to port 80 inside the Nginx container.

Docker allows you to stop the build process at a specific stage using the --target flag. If you want to build and test only the builder stage (ignoring the Nginx stage entirely)

docker build --target builder -t my-app-builder .

What is docker compose? Why is it used? How to write docker compose file?

Docker compose is a tool that allows you to define and run multiple containers applications using a single YAML file.

Docker Compose simplifies running multi-container applications by defining services, networks, and volumes in a single YAML file.

Problems it solves:

Without docker compose we have to-

1. Manually run multiple containers
2. Manage networks, ports, volumes separately
3. Hard to reproduce setup

With Docker compose we can

1. Define all services in a single file.
2. All containers start up together in the correct order.
3. They are placed on the same private “network” so they can talk to each other automatically (e.g., your Node app can reach your database just by calling http://database)..)
4. Your configuration is stored in code (YAML), making it easy to share with your team.
5. One-command start/stop
6. Ideal for local dev & testing

docker-compose.yml

version: "3.9"
services:
# Container 1: Our Node.js Application
 api:
    build: .                 # Tells Compose to build the Dockerfile in the current directory
    ports:
      - "8080:80"            # Maps localhost:8080 to container port 80
    depends_on:
      - database             # Ensures the database starts BEFORE this API container starts
    environment:
      - DB_HOST=database     # Passes environment variables to your code

# Container 2: The PostgreSQL Database
 database:
    image: postgres:15       # We don't need a Dockerfile for this, just use the official image
    ports:
      - "5432:5432"          # Optional: Exposes the DB to your local machine for debugging
    environment:
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: supersecretpassword
 networks:
 - app_net
networks:
 app_net:
    driver: bridge

Commands:

docker compose up # Start services
docker compose up -d # Detached mode
docker compose down # Stop & remove
docker compose ps # List services
docker compose logs # View logs

Key Concepts

Services

• Each service = one container
• Defined under services

Networks (Auto-created)

• Services communicate via service name

Volumes

volumes:
 - db_data:/var/lib/mysql

• Data persistence
• Named or bind volumes

Environment Variables

environment:
 APP_ENV: production

build vs image

build: .
image: myapp:latest

• build → build from Dockerfile
• image → pull from registry
• Can use both together

depends_on

depends_on:
 - db

• Controls startup order
• Does NOT wait for readiness
• Use healthcheck for real dependency

How Networking works in Docker?

Docker networking allows containers to communicate with each other, with hosts and the outside world(Internet) while being isolated by default.

By default, containers gets it own ip, runs on isolated network namespace and they can’t communicate with each other unless connected via a network.

Container
 ↕ (virtual ethernet)
Docker Network
 ↕
Host Network
 ↕
Internet

Types of network driver in docker

1. Bridge
2. Host
3. User-Defined Bridge
4. None

BRIDGE

By default, whenever you run a container without specifying a network, Docker attaches it to the default bridge network. Bridge network is like a virtual, private router inside your computer.

• Containers get private IPs (172.x.x.x)
• Can communicate by IP
• Need port mapping for host access

The Catch: On the default bridge network, containers can only find each other by their IP addresses. Because IP addresses change every time a container restarts, this is very unreliable for production applications.

docker network ls

Used For-

• running a single container for local testing

User Defined Bridge

To solve the changing IP address problem, we should use User Defined Bridge networks. When containers are connected with User Defined Bridge networks, they no longer need to know each others IP addresses, they can simply talk with each other using their container names only.

• Containers communicate by container name
• Built-in DNS
• Better isolation than default bridge

For example, if you have a database container named db and a web server container named web on the same user-defined network, the web container can connect to the database simply by sending traffic to the hostname db.

In case of docker compose, it automatically creates user defined bridge network for us. So the services can connect with each other just by service name

docker network create app_net

Used For-

• Running a standard multi-container application

Host

When we use host network stack, we do not need to publish port to host machine. With host, container shares same network as host. Meaning if the container run a web server at port 80 and we use docker run -- network host , the container’s application port is instantly available on port 80 of your actual host machine. We don’t need to publish port explicitly.

• Container shares host’s network stack
• No port mapping needed
• No isolation

docker run --network host imageName

Used For-

• Extremely high-throughput, low-latency applications (like a high-frequency trading bot or a massive traffic load balancer).
• Monitoring agents (like Prometheus Node Exporter) that specifically need to monitor the host machine’s actual network traffic.

None

This is completely isolated network. This driver completely disables all networking on the container. It gets no IP and can’t connect to other container or the internet.

docker run --network none busybox

Used For-

• Maximum-security, air-gapped tasks (e.g., a container that takes a local file, encrypts it using a secure key, and outputs the encrypted file).
• Running strictly local batch-processing jobs that have no business talking to the internet.

Docker Volumes

Generally when a container dies, its data also gets deleted by default as containers are ephemeral. Docker volume is a mechanism to persist data outside the container’s lifecycle. So even it container dies, data stays.

Without volumes- When container stops

• logs gone
• DB data gone
• app state lost

With volumes-

• Data stored outside containers
• Containers can be recreated safely
• Decouples data from application

docker volume create mydata
docker run -v mydata:/app/data myapp

mydata → volume name
/app/data → path inside container

# In docker Compose
services:
 db:
    image: mysql:8
    volumes:
    # Syntax:  host_path : container_path
      - db_data:/var/lib/mysql
volumes:
 db_data:

Docker Commands

# Docker Basics / Info
docker version # Docker client & server versions
docker info # System-wide info (storage, driver, limits)
docker help # All commands
docker <cmd> --help # Command-specific help

# Container Lifecycle (Creation, Updating, Deletion)

docker run [image]: Creates and starts a container from an image.
	-d: Run in detached mode (background).
	-p [host]:[container]: Publish/map a port.
	--name [name]: Assign a custom name to the container.
	-e [KEY=VALUE]: Pass environment variables.
	-v [volume/path]:[container_path]: Mount a volume or local directory.
	--network [network_name]: Connect to a specific network.
	--restart [policy]: Set restart rules (e.g., always, unless-stopped).
	--rm: Automatically remove the container when it exits.
	-it: Run interactively (allocates a pseudo-TTY and keeps STDIN open).

docker start [container]: Starts an already created, stopped container.

docker stop [container]: Gracefully stops a running container (sends SIGTERM).
	-t [seconds]: Seconds to wait for stop before killing it (default is 10).

docker kill [container]: Forcefully stops a container immediately (sends SIGKILL).
docker restart [container]: Stops and then starts a container.
docker rm [container]: Deletes a container (must be stopped first).
	-f: Force the removal of a running container.
	-v: Remove anonymous volumes associated with the container.

docker update [container]: Updates resource configurations of a running container.
	--cpus [value]: Update CPU limits.
	-m [value]: Update memory limits (e.g., 512m).

# Execution, Debugging & Inspection
docker exec [container] [command]: Runs a new command in a currently running container.
	-it: Run interactively (crucial for opening shells: docker exec -it my_app /bin/bash).
	-u [user]: Run the command as a specific user (e.g., -u root).
	-d: Run the command in the background.

docker logs [container]: Fetches the logs of a container.
	-f: Follow log output live.
	--tail [number]: Show only the last X number of lines.
	-t: Show timestamps.

docker inspect [container_or_image]: Returns low-level information (JSON format) on Docker objects.
	-f '[format_string]': Filter output using Go templates (e.g., docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' my_app).

docker ps (or docker container ls): Lists containers.
	-a: Show all containers (default shows just running).
	-q: Only display container IDs.

docker top [container]: Displays the running processes inside a container.

docker stats: Displays a live stream of container resource usage statistics (CPU, RAM, Network I/O).

# Networking
docker network create [network]: Creates a new user-defined network.
	-d [driver]: Specify the driver (e.g., bridge, overlay, macvlan). Default is bridge.

docker network ls: Lists all networks.
docker network inspect [network]: Shows detailed information on one or more networks.
docker network connect [network] [container]: Connects a running container to a network.
docker network disconnect [network] [container]: Disconnects a container from a network.
docker network rm [network]: Deletes one or more networks.

# Volumes & Mounts
docker volume create [volume]: Creates a managed volume.
docker volume ls: Lists all managed volumes.
docker volume inspect [volume]: Shows detailed information on a volume (including its exact mount point on the host).
docker volume rm [volume]: Deletes a volume.
 -f: Force removal.
 (Mounting via docker run):
 -v: Older, simpler syntax (-v host_path:container_path:ro for read-only).
 --mount: Newer, more explicit syntax (--mount type=bind,source=/host,target=/container).

# System & Cleanup Operations
docker system prune: Cleans up unused data. This is the ultimate cleanup command.
 -a: Remove all unused images, not just dangling ones.
 --volumes: Remove all unused volumes as well.
 -f: Bypass the confirmation prompt.

docker image prune / docker container prune / docker volume prune / docker network prune: Targets specific unused resources for cleanup.
docker info: Displays system-wide information (number of containers, images, storage driver, etc.).
docker history [image]: Shows the history/layers of an image.