File System Full

File System Full

Pada suatu pagi di hari minggu, lagi enak-enaknya minum kopi ada telp
disuruh cek partisi pada salah satu router yang menurut informasi
terjadi kejanggalan dikatakan bahwa partisi /var full.

Tanpa basa-basi langsung cek n ricek ke router tujuan



Cek n Ricek

$ sudo tail -f /var/log/messages
Nov 22 08:56:58 GreenRouter-JKT01 /bsd: uid 1000 on /var: file system full
Nov 22 08:57:04 GreenRouter-JKT01 /bsd: uid 0 on /var: file system full
Nov 22 08:57:04 GreenRouter-JKT01 pflogd[11142]: Logging suspended: fwrite: No space left on device
Nov 22 08:58:05 GreenRouter-JKT01 /bsd: uid 0 on /var: file system full
Nov 22 08:58:05 GreenRouter-JKT01 pflogd[11142]: Logging suspended: fwrite: No space left on device
Nov 22 08:59:05 GreenRouter-JKT01 /bsd: uid 0 on /var: file system full
Nov 22 08:59:05 GreenRouter-JKT01 pflogd[11142]: Logging suspended: fwrite: No space left on device

$ df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/wd0a 4.9G 117M 4.6G 2% /
/dev/wd0g 15.3G 74.0M 14.4G 0% /data
/dev/wd0e 9.8G 2.0K 9.4G 0% /tmp
/dev/wd0d 19.7G 2.3G 16.4G 12% /usr
/dev/wd0f 19.7G 19.7G -1007M 105% /var

$ sudo du -h -s /var/
109M /var/

Wah ini aneh hasil yang diberikan df -h dan du -h -s berbeda...

Cek lebih jauh lagi

$ fstat -f /var
USER CMD PID FD MOUNT INUM MODE R/W SZ|DV
root ksh 20619 wd /var 2 drwxr-xr-x r 512
asoel ksh 17578 wd /var 2 drwxr-xr-x r 512
asoel ksh 18941 wd /var 2 drwxr-xr-x r 512
root systat 27253 3 /var 675585 -rw-rw-r-- r 8100
root systat 27253 6 /var 1273220 -rw-r--r-- r 1294336
root sendmail 21369 wd /var 2494471 drwx------ r 512
root sendmail 21369 8 /var 675595 -rw------- w 70
root cron 6171 wd /var 1714944 dr-xr-xr-x r 512
root cron 6171 3 /var 675594 -rw-r--r-- rw 5
_bgpd bgpd 18359 wd /var 1818880 drwxr-xr-x r 512
_bgpd bgpd 18359 root /var 1818880 drwxr-xr-x r 512
_bgpd bgpd 21279 wd /var 1818880 drwxr-xr-x r 512
_bgpd bgpd 21279 root /var 1818880 drwxr-xr-x r 512
_snmpd snmpd 2308 wd /var 1818880 drwxr-xr-x r 512
_snmpd snmpd 2308 root /var 1818880 drwxr-xr-x r 512
_ntp ntpd 24212 wd /var 1818880 drwxr-xr-x r 512
_ntp ntpd 24212 root /var 1818880 drwxr-xr-x r 512
_pflogd pflogd 11142 wd /var 1818880 drwxr-xr-x r 512
_pflogd pflogd 11142 root /var 1818880 drwxr-xr-x r 512
_pflogd pflogd 11142 4 /var 2442524 -rw------- rw 21050293545
_syslogd syslogd 498 wd /var 1818880 drwxr-xr-x r 512
_syslogd syslogd 498 root /var 1818880 drwxr-xr-x r 512
_syslogd syslogd 498 9 /var 2442508 -rw-r----- w 9019
_syslogd syslogd 498 11 /var 2442530 -rw------- w 11154
_syslogd syslogd 498 12 /var 1714950 -rw------- w 5679
_syslogd syslogd 498 14 /var 2442509 -rw-r----- w 0
_syslogd syslogd 498 15 /var 2442503 -rw-r----- w 0
_syslogd syslogd 498 16 /var 2442552 -rw------- w 9518

Ternyata ada sesuatu yang aneh pada pflogd
_pflogd pflogd 11142 4 /var 2442524 -rw------- rw 21050293545

Dilakukan P3K takutnya keburu tewas routernya maklum itu router berada di Jakarta...

$ sudo pkill pflogd && sudo pflogd

Cek n Ricek lagi....

$ fstat -f /var
USER CMD PID FD MOUNT INUM MODE R/W SZ|DV
_pflogd pflogd 6702 wd /var 1818880 drwxr-xr-x r 512
_pflogd pflogd 6702 root /var 1818880 drwxr-xr-x r 512
_pflogd pflogd 6702 4 /var 2442512 -rw------- rw 10274624
asoel ksh 21256 wd /var 2 drwxr-xr-x r 512
asoel ksh 17578 wd /var 2 drwxr-xr-x r 512
asoel ksh 18941 wd /var 2 drwxr-xr-x r 512
root systat 27253 3 /var 675585 -rw-rw-r-- r 8100
root systat 27253 6 /var 1273220 -rw-r--r-- r 1294336
root sendmail 21369 wd /var 2494471 drwx------ r 512
root sendmail 21369 8 /var 675595 -rw------- w 70
root cron 6171 wd /var 1714944 dr-xr-xr-x r 512
root cron 6171 3 /var 675594 -rw-r--r-- rw 5
_bgpd bgpd 18359 wd /var 1818880 drwxr-xr-x r 512
_bgpd bgpd 18359 root /var 1818880 drwxr-xr-x r 512
_bgpd bgpd 21279 wd /var 1818880 drwxr-xr-x r 512
_bgpd bgpd 21279 root /var 1818880 drwxr-xr-x r 512
_snmpd snmpd 2308 wd /var 1818880 drwxr-xr-x r 512
_snmpd snmpd 2308 root /var 1818880 drwxr-xr-x r 512
_ntp ntpd 24212 wd /var 1818880 drwxr-xr-x r 512
_ntp ntpd 24212 root /var 1818880 drwxr-xr-x r 512
_syslogd syslogd 498 wd /var 1818880 drwxr-xr-x r 512
_syslogd syslogd 498 root /var 1818880 drwxr-xr-x r 512
_syslogd syslogd 498 12 /var 1714950 -rw------- w 5970


$ df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/wd0a 4.9G 117M 4.6G 2% /
/dev/wd0g 15.3G 78.1M 14.4G 1% /data
/dev/wd0e 9.8G 2.0K 9.4G 0% /tmp
/dev/wd0d 19.7G 2.3G 16.4G 12% /usr
/dev/wd0f 19.7G 86.8M 18.6G 0% /var

Akhirnya file system /var kembali ke semula....system masih berjalan normal.

Dilanjut lagi ah weekend nya....

Note :
Jika ada waktu luang fsck -f -y /var pada single user mode