Skip to content
Go back

Week 16: Batch Jobs & Scaling Inference

Edit page

🐳 Docker 360 β€” Week 16 of 16 | Batch jobs, scaling inference & graduation πŸŽ“

🎯 Learning Objective

Ye hamara aakhri week hai! Tum seekhoge ke one-shot batch containers se ETL-style jobs kaise chalaye jate hain, ek trained ML model ko API container ke peeche kaise serve kiya jata hai, aur replicas + load balancer se inference ko kaise scale kiya jata hai β€” ek simple load test ke sath. Aur end me poore 16-week safar ka wrap-up: tumne kya seekha aur agla qadam (Kubernetes, GitOps) kya hai.

🎨 Visual Diagrams

One-shot batch job ka lifecycle:

  cron / scheduler ──trigger──►  docker run --rm etl-job
                                              β”‚
                                              β–Ό
                                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                                    β”‚  container runs  β”‚
                                    β”‚  extract β†’       β”‚
                                    β”‚  transform β†’     β”‚
                                    β”‚  load β†’ exit 0   β”‚
                                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                              β”‚
                                     --rm: khud delete βœ“
                                              β”‚
                                              β–Ό
                                    logs me result, koi
                                    leftover container nahi

Scaled inference ke peeche load balancer:

                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  clients ─────────►│  nginx (load balancer, :8080)   β”‚
                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                             β”‚  round-robin
              β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
              β–Ό              β–Ό              β–Ό
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”   β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚ api-1    β”‚   β”‚ api-2    β”‚   β”‚ api-3    β”‚
        β”‚ :8000    β”‚   β”‚ :8000    β”‚   β”‚ :8000    β”‚
        β”‚ model 🧠 β”‚   β”‚ model 🧠 β”‚   β”‚ model 🧠 β”‚
        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
        (docker compose up --scale api=3)

🧠 Theory Deep Dive

One-shot containers (--rm)

Har container hamesha chalti hui service nahi hoti. Batch jobs β€” data processing, backups, nightly reports β€” chal kar khatam hone wale containers hain. --rm flag container ke exit hote hi use delete kar deta hai, taake docker ps -a me murda containers ka dher na lage. Scheduling ke liye host ka cron, systemd timer, ya orchestrator (Kubernetes CronJob) use hota hai β€” container khud schedule nahi karta, wo sirf kaam karta hai.

Model serving pattern

ML model ko serve karne ka standard tareeqa: model file (.pkl) ko image me bake karo ya volume se mount karo, ek lightweight API (FastAPI) usay load kare, aur /predict endpoint expose kare. Har replica stateless hoti hai β€” isi liye unhe scale karna aasan hai: kaam ka bojh kitna bhi ho, replicas barha do.

Scaling with Compose

docker compose up --scale api=3 ek service ke 3 containers bana deta hai. Ehm khayal rahe: scaled service par host port publish nahi kar sakte (3 containers ek hi host port ke liye larhenge). Isi liye aage nginx jaisa load balancer lagate hain β€” wo ek port sunta hai aur requests replicas me baant deta hai. Compose ke andar service naam (api) se DNS round-robin hota hai.

πŸ’» Hands-On Lab

Lab 1 β€” One-shot batch job

Step 1: Ek ETL-style script banao:

mkdir -p /tmp/batch-lab && cd /tmp/batch-lab
cat > etl.py <<'EOF'
import datetime, json

print("extracting...")
raw = [{"id": i, "value": i * 10} for i in range(1, 6)]

print("transforming...")
clean = [{"id": r["id"], "value": r["value"], "doubled": r["value"] * 2} for r in raw]

print("loading...")
with open("/output/result.json", "w") as f:
    json.dump({"ran_at": datetime.datetime.utcnow().isoformat(), "rows": clean}, f, indent=2)

print(f"done: {len(clean)} rows written")
EOF
mkdir -p output

Step 2: One-shot run:

docker run --rm -v /tmp/batch-lab:/work -v /tmp/batch-lab/output:/output -w /work python:3.12-slim python etl.py
cat output/result.json

βœ… Expected result: done: 5 rows written aur result.json me 5 transformed rows. Phir docker ps -a me koi leftover container nahi milega β€” --rm ne safai kar di.

Step 3: Scheduling note β€” isay nightly chalana ho to host cron me:

# crontab -e me ye line (roz raat 2 baje):
# 0 2 * * * docker run --rm -v /tmp/batch-lab:/work -v /tmp/batch-lab/output:/output -w /work python:3.12-slim python etl.py >> /var/log/etl.log 2>&1

Lab 2 β€” Model training (pickle banao)

Step 1: (Host par pip install scikit-learn chahiye)

mkdir -p /tmp/serve-lab/app && cd /tmp/serve-lab
cat > train_model.py <<'EOF'
import pickle
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression

X, y = load_iris(return_X_y=True)
model = LogisticRegression(max_iter=200)
model.fit(X, y)
with open("app/model.pkl", "wb") as f:
    pickle.dump(model, f)
print("model saved -> app/model.pkl")
EOF
python train_model.py

Lab 3 β€” API container

Step 2: app/main.py:

from fastapi import FastAPI
from pydantic import BaseModel
import pickle, socket

with open("model.pkl", "rb") as f:
    model = pickle.load(f)

app = FastAPI()
host = socket.gethostname()

class Features(BaseModel):
    features: list[float]

@app.post("/predict")
def predict(payload: Features):
    pred = model.predict([payload.features])[0]
    return {"prediction": int(pred), "served_by": host}

@app.get("/health")
def health():
    return {"status": "ok", "served_by": host}

Step 3: app/requirements.txt:

fastapi
uvicorn[standard]
scikit-learn

Step 4: Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY app/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Lab 4 β€” Scale + load balancer

Step 5: nginx.conf:

events {}

http {
    upstream api {
        server api:8000;
    }

    server {
        listen 8080;

        location / {
            proxy_pass http://api;
        }
    }
}

Step 6: compose.yaml:

services:
  api:
    build: .
    expose:
      - "8000"
  nginx:
    image: nginx:alpine
    ports:
      - "8080:8080"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api

Step 7: 3 replicas ke sath up karo:

docker compose up --build -d --scale api=3
sleep 10
docker compose ps

βœ… Expected result: api-1, api-2, api-3 aur nginx-1 β€” 4 containers running.

Step 8: Inference test:

curl -s -X POST localhost:8080/predict -H 'Content-Type: application/json' \
  -d '{"features":[5.1,3.5,1.4,0.2]}'

βœ… Expected result: {"prediction":0,"served_by":"..."} β€” model ne jawab diya!

Lab 5 β€” Simple load test: distribution dekho

Step 9: 60 requests bhejo aur dekho load bant-ta hai ya nahi:

for i in $(seq 1 60); do
  curl -s -X POST localhost:8080/predict -H 'Content-Type: application/json' \
    -d '{"features":[5.1,3.5,1.4,0.2]}'
  echo
done | grep -o '"served_by":"[^"]*"' | sort | uniq -c

βœ… Expected result: Teenon replicas me lagbhag barabar taqseem β€” jaise 20 api-1, 20 api-2, 20 api-3 (thoda upar neeche ho sakta hai). Yehi load balancing ka saboot hai: nginx requests ko round-robin baant raha hai.

Step 10: Scale down karke farq dekho:

docker compose up -d --scale api=1
sleep 5
docker compose ps --format '{{.Name}}'

βœ… Expected result: Sirf ek api container reh jayega β€” scaling dono direction me kaam karti hai.

Step 11: Safai:

docker compose down --rmi local -v

βœ… Checkpoint

Q1: Batch job containers me --rm kyun lagate hain? A: Taake job complete hote hi container auto-delete ho jaye β€” warna har run ek murda container chhor jayega aur docker ps -a bhar jayega.

Q2: Scaled service (--scale api=3) par host port publish kyun nahi kar sakte? A: Kyunke 3 containers ek hi host port ke liye larhenge β€” port conflict hoga. Isi liye aage load balancer lagate hain jo ek port sunta hai.

Q3: Load test me served_by field ka kya maqsad tha? A: Ye batata hai ke kaunsi replica ne request handle ki β€” is se verify hota hai ke nginx requests ko replicas me baant raha hai, sab ek par nahi aa rahe.

Q4: Inference replicas ko stateless rakhna kyun zaroori hai? A: Taake koi bhi replica kisi bhi request ko handle kar sake β€” phir replicas add/remove karna safe hai aur load balancer ko koi farq nahi parhta.

πŸŽ“ Course Wrap-Up β€” 16 Weeks Complete!

Mubarak ho! πŸŽ‰ Tumne Docker 360 ka poora safar tay kar liya. Ek nazar peeche:

WeeksKya seekha
1–4Docker basics β€” containers vs VMs, images, Dockerfile, CLI
5–8Volumes, bind mounts, networking, Compose multi-container apps
9–12Registries, security best practices, CI pipelines, monitoring
13Docker-in-Docker workflows aur CI builds
14Team dev environments, override files, seeded databases
15GPU containers, CUDA images, MLflow experiment tracking
16Batch jobs, model serving, scaling inference

Tum ab containers chalana, banana, secure karna, scale karna aur ML workloads par lagana β€” sab jante ho. Ye wo skill set hai jo real DevOps roles me roz kaam aata hai.

Agla qadam β€” yahan se kahan?

Seekhte raho, labs karte raho β€” aur yaad rakho: β€œJo samjhay wohi sikhtay hain.” Shukriya ke tum is safar me sath rahe! πŸš€πŸ³


Edit page
Share this post on:

Next Post
Week 15: GPU Containers & MLflow