*File System & Disk Management**
1. **Scenario**: `df` shows 100% disk usage, but `du -sh /` reports only 60% usage. What’s happening?
**Fix**: `lsof +L1 | grep deleted` → Find processes holding deleted files → Restart affected services.
2. **Scenario**: `rm -rf /data` fails with “Argument list too long”. How to fix?
**Fix**: `find /data -type f -delete` or `rsync -a –delete empty_dir/ /data/`
3. **Scenario**: Ext4 filesystem corruption after power failure. How to repair?
**Fix**: `umount /dev/sdb1 && fsck -y /dev/sdb1`
4. **Scenario**: NFS client hangs when server reboots. How to prevent system freeze?
**Fix**: Mount with `hard,intr` options: `mount -o hard,intr nfsserver:/share /mnt`
5. **Scenario**: Accidentally ran `chmod -x /bin/chmod`. How to recover?
**Fix**: `cp /bin/chmod /tmp/chmod; /tmp/chmod +x /bin/chmod`
*Networking**
6. **Scenario**: Can ping server but SSH connection times out.
**Fix**: Check firewall: `iptables -L -n -v` or `firewall-cmd –list-all`
7. **Scenario**: `Device eth0 does not exist` after reboot.
**Fix**: Rename interface: `vi /etc/udev/rules.d/70-persistent-net.rules`
8. **Scenario**: DNS resolution slow. How to diagnose?
**Fix**: `time dig example.com` → Check `/etc/resolv.conf` → Test with `tcpdump -i any port 53`
9. **Scenario**: Port 8080 open but application not reachable.
**Fix**: Check SELinux: `setsebool -P httpd_can_network_connect=1`
10. **Scenario**: Multicast traffic not reaching server.
**Fix**: Enable multicast route: `ip route add 224.0.0.0/4 dev eth0`
*Process Management**
11. **Scenario**: `fork: Cannot allocate memory` but free RAM exists.
**Fix**: Check process limit: `ulimit -u` → Increase with `sysctl kernel.pid_max`
12. **Scenario**: Zombie processes accumulating.
**Fix**: `ps aux | grep ‘Z’` → Kill parent process: `kill -HUP PPID`
13. **Scenario**: Process hangs during `kill -9`.
**Fix**: Check kernel wait: `cat /proc/PID/stack` → Unmount stuck filesystems.
14. **Scenario**: `sudo` commands hang for 30 seconds.
**Fix**: Disable reverse DNS in `/etc/sudoers`: `Defaults !fqdn`
*Boot & Kernel**
15. **Scenario**: System boots to `initramfs` prompt after disk failure.
**Fix**: `fsck /dev/sda1` → `exit` to continue boot.
16. **Scenario**: “Kernel panic – not syncing: VFS: Unable to mount root fs”
**Fix**: Check initrd: `mkinitrd -f /boot/initramfs-$(uname -r).img $(uname -r)`
17. **Scenario**: Missing module after kernel upgrade.
**Fix**: `dracut –regenerate-all -f` or `update-initramfs -u`
*Security*
18. **Scenario**: User can’t delete file despite 777 permissions.
**Fix**: Check immutable flag: `lsattr file` → `chattr -i file`
19. **Scenario**: SSH login rejected after too many failed attempts.
**Fix**: `pam_tally2 -u username -r` to reset counter.
20. **Scenario**: Apache fails to bind to port 80 as non-root.
**Fix**: `setcap ‘cap_net_bind_service=+ep’ /usr/sbin/httpd`
**Performance**
21. **Scenario**: High `wa` in `top` but low disk utilization.
**Fix**: Check I/O scheduler: `cat /sys/block/sda/queue/scheduler` → Switch to `deadline`
22. **Scenario**: MySQL slow queries during peak hours.
**Fix**: `pidstat -d 1` → Identify high-IO process → Optimize queries.
23. **Scenario**: Random system freezes for 10-20 seconds.
**Fix**: Check NMI watchdog: `dmesg | grep NMI` → `sysctl kernel.nmi_watchdog=0`
**Package Management**
24. **Scenario**: `apt-get update` fails with `Hash Sum mismatch`.
**Fix**: `rm -rf /var/lib/apt/lists/*` → `apt-get clean` → Retry.
25. **Scenario**: Dependency hell during RPM upgrade.
**Fix**: `rpm -Uvh –nodeps package.rpm` (last resort) or use `yum history undo`
**Logging & Monitoring**
26. **Scenario**: `journalctl` shows “No journal files were found”.
**Fix**: `mkdir -p /var/log/journal` → `systemctl restart systemd-journald`
27. **Scenario**: `/var/log/messages` missing after reboot.
**Fix**: Check `rsyslog` status → Enable persistent logging in `/etc/rsyslog.conf`
*Advanced Troubleshooting**
28. **Scenario**: Server unreachable via SSH but console works.
**Fix**: Check MTU: `ip link show eth0` → `ifconfig eth0 mtu 1400`
29. **Scenario**: Random process crashes with `Segmentation fault`.
**Fix**: `ulimit -c unlimited` → Analyze core dump with `gdb`.
30. **Scenario**: Time drifts in VM causing certificate errors.
**Fix**: Enable NTP: `chronyc makestep` → Disable host time sync.
**Container/Docker**
31. **Scenario**: Docker container can’t resolve DNS.
**Fix**: Edit `/etc/docker/daemon.json`: `{“dns”: [“8.8.8.8”]}`
32. **Scenario**: `docker build` fails with `no space left on device`.
**Fix**: Prune: `docker system prune -af` → Clean `/var/lib/docker`
**Cloud Environments**
33. **Scenario**: AWS instance loses network after stop/start.
**Fix**: Remove persistent net rules: `rm /etc/udev/rules.d/70-persistent-net.rules`
34. **Scenario**: Cloud-init fails during provisioning.
**Fix**: Check logs: `/var/log/cloud-init.log` → Repair with `cloud-init clean –logs`
*Shell & CLI**
35. **Scenario**: `Ctrl+C` doesn’t terminate running command.
**Fix**: Use `Ctrl+\` (SIGQUIT) or `kill -9 PID` from another terminal.
36. **Scenario**: Script fails with `[: too many arguments`.
**Fix**: Quote variables: `if [ “$var” = “value” ]` instead of `[ $var = value ]`
**Database Servers**
37. **Scenario**: PostgreSQL won’t start after abrupt shutdown.
**Fix**: `pg_resetwal /var/lib/pgsql/data` (WARNING: potential data loss)
38. **Scenario**: MongoDB high memory usage.
**Fix**: Limit cache: `mongod –wiredTigerCacheSizeGB 2`
### **Security Scenarios**
39. **Scenario**: `sudo` reports “user not in sudoers file” for valid user.
**Fix**: `visudo` → Ensure group syntax: `%sudo ALL=(ALL) ALL`
40. **Scenario**: SSH login works with key but fails with password.
**Fix**: Edit `/etc/ssh/sshd_config`: `PasswordAuthentication yes`
**Critical Production Scenarios (Senior Level)**
41. **Scenario**: `rm -rf /*` executed accidentally. What immediate actions?
**Fix**: Unmount filesystems → Restore from backup → Use `extundelete` for recovery.
42. **Scenario**: Database disk full during transaction.
**Fix**: Create emergency space: `dd if=/dev/zero of=/tmp/emergency bs=1M count=1000; losetup` → Expand disk.
43. **Scenario**: RAID degradation detected. How to replace failed disk?
**Fix**: `mdadm –manage /dev/md0 –fail /dev/sdb1` → `mdadm –manage /dev/md0 –remove /dev/sdb1` → Add new disk.
44. **Scenario**: Kernel OOM killer terminates critical process.
**Fix**: Prioritize processes: `echo -1000 > /proc/PID/oom_score_adj`
45. **Scenario**: `fsck` requires manual intervention during boot.
**Fix**: Answer prompts carefully → Use `-y` flag in future automated checks.
46. Recover encrypted LVM after `/etc/lvm` corruption: `vgcfgrestore`
47. Fix “Stale file handle” NFS error: `umount -f /mnt; umount -l /mnt`
48. Diagnose “TCP: too many orphaned sockets”: `sysctl net.ipv4.tcp_max_orphans`
49. Resolve “Address already in use” for closed port: `ss -tulp` → Kill zombie process
50. Fix “No such file or directory” for existing file: Check filesystem with `dmesg`
51. Repair broken package: `dpkg –configure -a` or `rpm –rebuilddb`
52. Fix “Argument list too long” for `tar`: Use `find . -print0 | tar -czvf backup.tar.gz –null -T -`
53. Resolve “Disk quota exceeded” despite free space: `repquota -a` → Increase inode limit
54. Fix “Cannot open display” for X11: `xhost +` and check `$DISPLAY`
55. Diagnose random high CPU: `perf top` or `strace -c -p PID`