Skip to content
Go back

Week 3: Persistent Storage Mastery

Edit page

🐳 Docker 360 β€” Week 3 of 16 | Persistent storage: volumes, bind mounts, tmpfs aur PostgreSQL ka production setup

🎯 Learning Objective

Week 1 me hum ne dekha tha: container ka writable layer ephemeral hai β€” container gaya, data gaya. Real duniya me databases, uploads aur configs ko persist karna parta hai. Is week me tum teeno storage options (volumes, bind mounts, tmpfs) ke darmiyan sahi faisla karna seekho ge, volume backup/restore ka lab karo ge, aur PostgreSQL ko named volume + init scripts + healthcheck ke sath production style me chalao ge.

🎨 Visual Diagrams

Storage options β€” data kahan rehta hai?

  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚                        HOST MACHINE                         β”‚
  β”‚                                                             β”‚
  β”‚  /var/lib/docker/volumes/        /home/user/project/        β”‚
  β”‚   └─ pgdata/                     └─ config/                 β”‚
  β”‚       └─ _data/  ◄── VOLUME           └─ app.conf ◄── BIND  β”‚
  β”‚            β”‚  (Docker manage                          MOUNT β”‚
  β”‚            β”‚   karta hai)                        (tum manageβ”‚
  β”‚            β”‚                                      karte ho)β”‚
  β”‚            β–Ό                                               β”‚
  β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”           β”‚
  β”‚  β”‚   Container      β”‚        β”‚   Container      β”‚           β”‚
  β”‚  β”‚  /var/lib/       β”‚        β”‚  /etc/app/       β”‚           β”‚
  β”‚  β”‚  postgresql/data β”‚        β”‚   app.conf       β”‚           β”‚
  β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜           β”‚
  β”‚                                                             β”‚
  β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                       β”‚
  β”‚  β”‚   Container      β”‚   tmpfs: sirf RAM me, container       β”‚
  β”‚  β”‚  /run/secrets ── β”‚   stop hote hi data GONE. Secrets     β”‚
  β”‚  β”‚  (RAM only!)     β”‚   aur temp caches ke liye.            β”‚
  β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                       β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Volume backup & restore flow

  BACKUP                                    RESTORE
  ======                                    =======

  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚ volume       β”‚                          β”‚ tarball      β”‚
  β”‚ "appdata"    β”‚                          β”‚ appdata.tgz  β”‚
  β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚   tar czf                β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚   tar xzf
  β”‚  β”‚notes.txt│──┼────────► appdata.tgz ───┼─►│notes.txt│──┼────────►
  β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚                          β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚   new volume
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   "appdata2"

  Helper container:                         Helper container:
  --rm -v appdata:/data                     --rm -v appdata2:/data
  -v $(pwd):/backup                         -v $(pwd):/backup
  alpine tar czf /backup/...                alpine tar xzf /backup/...

  Trick ye hai ke backup ke liye app container ko rokne ki
  zaroorat nahi β€” ek chhota helper container volume mount
  karke tar bana leta hai.

PostgreSQL lab architecture

  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚  docker compose (week3-lab)                      β”‚
  β”‚                                                  β”‚
  β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                  β”‚
  β”‚  β”‚ db (postgres:16-alpine)    β”‚                  β”‚
  β”‚  β”‚  :5432 (internal only)     β”‚                  β”‚
  β”‚  β”‚                            β”‚                  β”‚
  β”‚  β”‚  /var/lib/postgresql/data ─┼──► volume: pgdataβ”‚
  β”‚  β”‚  /docker-entrypoint-        β”‚    (PERSISTENT!) β”‚
  β”‚  β”‚   initdb.d/init.sql ───────┼──► bind mount    β”‚
  β”‚  β”‚                            β”‚    ./init.sql    β”‚
  β”‚  β”‚  healthcheck: pg_isready   β”‚    (first boot   β”‚
  β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     par chalta   β”‚
  β”‚                                      hai)        β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

  compose down karne par bhi pgdata volume rehta hai β€”
  dubara up karne par data wapas milta hai. Iska matlab
  ye hai ke database containers ke sath "mar" nahi jati.

🧠 Theory Deep Dive

Decision guide β€” kab kya use karo?

VolumesBind mountstmpfs
Manage kaun karta haiDocker (/var/lib/docker/volumes/)Tum (koi bhi host path)Kernel (RAM)
Best forDatabases, app data β€” production defaultDev me live code reload, config filesSecrets, temp caches
BackupEasy (helper container + tar)Normal file copyN/A β€” RAM hai
PerformanceStorage driver bypass β€” fastHost filesystem speedSab se tez (RAM)
PortabilityCompose/swarm me bestHost path har machine par hona chahiyeSirf Linux
Example-v pgdata:/var/lib/postgresql/data-v ./src:/app--tmpfs /run/secrets

Rule of thumb: Production data β†’ named volumes. Development me code edit karna β†’ bind mounts. Sensitive temp data β†’ tmpfs. Random host paths par data rakhna (anonymous volumes se bacho β€” docker volume ls me dangling volumes ka dher lag jata hai).

Named vs anonymous volumes

docker run -d -v pgdata:/var/lib/postgresql/data postgres:16-alpine   # NAMED βœ“
docker run -d -v /var/lib/postgresql/data postgres:16-alpine          # ANONYMOUS βœ—

Anonymous volumes ko random hash naam milta hai β€” container delete hone ke baad wo orphan reh jate hain aur pata nahi chalta ke kis ka data tha. Hamesha naam do.

Init scripts β€” database ka pehla din

Postgres (aur MySQL) images ka entrypoint /docker-entrypoint-initdb.d/ folder me rakhi .sql, .sh ya .sql.gz files ko sirf pehli baar (jab data directory khaali ho) chalata hai. Iska matlab ye hai ke schema aur seed data version-controlled files me rehta hai β€” container dubara banao, database khud tayyar.

Healthchecks β€” β€œrunning” vs β€œready”

Container running hai iska matlab ye nahi ke app ready hai β€” database ko 5-10 second lagte hain. Healthcheck Docker ko batata hai ke asal me service healthy hai ya nahi:

healthcheck:
  test: ["CMD-SHELL", "pg_isready -U labuser -d labdb"]
  interval: 5s
  timeout: 3s
  retries: 5
  start_period: 10s

Compose me doosri services depends_on with condition: service_healthy laga kar database ke ready hone ka wait kar sakti hain β€” race conditions khatam.

Storage performance notes

πŸ’» Hands-On Lab

Lab A β€” Volume backup & restore

Step 1: Named volume banao aur us me data likho.

docker volume create appdata
docker run --rm -v appdata:/data alpine sh -c 'echo "ye mera important data hai" > /data/notes.txt && echo "doosri line" >> /data/notes.txt'
docker run --rm -v appdata:/data alpine cat /data/notes.txt

βœ… Expected result: Dono lines print hoti hain β€” data volume me persist hai, container to kab ka delete ho chuka (--rm).

Step 2: Helper container se backup (tar) banao.

mkdir -p ~/week3-lab && cd ~/week3-lab
docker run --rm -v appdata:/data -v "$(pwd)":/backup alpine tar czf /backup/appdata-backup.tar.gz -C /data .
ls -lh appdata-backup.tar.gz

βœ… Expected result: appdata-backup.tar.gz file banti hai. App container ko rokna nahi para β€” helper ne volume mount karke backup le liya.

Step 3: Disaster simulate karo β€” volume delete!

docker volume rm appdata
docker volume ls | grep appdata || echo "volume gaya!"

βœ… Expected result: Volume list me nahi hai. (Real life me yehi woh lamha hai jab dil ki dharkan tez hoti hai.)

Step 4: Restore karo β€” naya volume, backup se data wapas.

docker volume create appdata-restored
docker run --rm -v appdata-restored:/data -v "$(pwd)":/backup alpine tar xzf /backup/appdata-backup.tar.gz -C /data
docker run --rm -v appdata-restored:/data alpine cat /data/notes.txt

βœ… Expected result: Dono lines wapas! Backup/restore ka poora cycle kamyab. Ghabrana nahi tha β€” backup tha na.

Lab B β€” PostgreSQL: persistent + init scripts + healthcheck

Step 5: Init script likho.

cat > init.sql <<'EOF'
CREATE TABLE IF NOT EXISTS students (
  id SERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  city TEXT
);
INSERT INTO students (name, city) VALUES
  ('Ali', 'Lahore'),
  ('Fatima', 'Karachi')
ON CONFLICT DO NOTHING;
EOF

Step 6: Compose file banao.

cat > compose.yaml <<'EOF'
services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: labuser
      POSTGRES_PASSWORD: labpass123
      POSTGRES_DB: labdb
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U labuser -d labdb"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 10s

volumes:
  pgdata:
EOF
docker compose up -d

βœ… Expected result: Container start hota hai. Pehli baar init script chal kar table + seed data banata hai.

Step 7: Healthy hone ka wait karo aur data verify karo.

sleep 12
docker compose ps
docker exec week3-lab-db-1 psql -U labuser -d labdb -c "SELECT * FROM students;"

βœ… Expected result: ps me status healthy (running ke sath), aur query me Ali/Fatima ki rows. (Container ka naam docker compose ps se confirm kar lo β€” folder ke naam par mabni hota hai.)

Step 8: Naya data insert karo, phir container ura do.

docker exec week3-lab-db-1 psql -U labuser -d labdb -c "INSERT INTO students (name, city) VALUES ('Usman', 'Islamabad');"
docker compose down
docker compose up -d
sleep 12
docker exec week3-lab-db-1 psql -U labuser -d labdb -c "SELECT * FROM students;"

βœ… Expected result: Teeno rows maujood β€” down ke baad bhi data mehfooz, kyun ke wo volume pgdata me hai, container me nahi. Yehi volumes ka asal power hai.

Step 9: Healthcheck ka status dekho.

docker inspect week3-lab-db-1 --format='{{json .State.Health.Status}}'

βœ… Expected result: "healthy" β€” Docker khud monitor kar raha hai ke postgres asal me jawab de raha hai.

Cleanup:

cd ~/week3-lab && docker compose down -v
docker volume rm appdata-restored
cd ~ && rm -rf ~/week3-lab

βœ… Checkpoint

Q1: Named volumes, bind mounts aur tmpfs me se database ke liye kya best hai aur kyun? A: Named volumes β€” Docker inko manage karta hai, ye storage driver ko bypass karte hain (behtar I/O performance), portable hain, aur backup/restore easy hai. Bind mounts dev ke liye, tmpfs RAM-only temp data ke liye.

Q2: Anonymous volumes se kyun bachna chahiye? A: Inko random hash naam milta hai β€” container delete hone ke baad orphan volumes reh jate hain, pata nahi chalta ke data kis ka tha. docker volume ls kachre se bhar jata hai.

Q3: /docker-entrypoint-initdb.d/ me rakhi SQL files kab chalti hain? A: Sirf pehli baar, jab data directory bilkul khaali ho. Pehle se initialized volume par dubara nahi chalti β€” is liye schema changes ke liye migrations use karo.

Q4: Healthcheck depends_on ke sath mil kar kya problem solve karta hai? A: Race condition β€” app container database ke ready hone se pehle start ho kar crash ho jata hai. Healthcheck + service_healthy condition se dependent service tab hi start hoti hai jab database asal me connections accept kar raha ho.

πŸ”œ Next Week

Week 4 me hum single container se bahar niklenge β€” Docker Swarm par cluster banao, services ko scale karo, rolling updates aur auto-recovery ka lab karo. Orchestration ki duniya me khush aamdeed!


Edit page
Share this post on:

Previous Post
Week 4: Orchestration Fundamentals
Next Post
Week 2: Advanced Docker Networking