How do I find out more about socket files in /proc/fd

How do I find out more about socket files in /proc/fd?

When debugging connections you will sometimes see a socket is timing out or hanging causing issues.

Example

Having a hanging php-fpm process that is causing max_children

First thing you will want to do is grab the longest running php-fpm process.

ps aux --sort=start_time | grep ^(useraccount) | grep php-fpm

Sample Output:

nelsonweb 47029  2.1  0.8 916744 213880 ?       SN   03:30   8:52 php-fpm: pool nelsonweb
nelsonweb 11079 21.0  0.8 930156 214416 ?       RN   10:02   3:11 php-fpm: pool nelsonweb

Now you will want to get that pid and run an strace on it

strace -p 47029

Sample Output:

Process 47029 attached
restart_syscall(<... resuming interrupted call ...>) = 0 
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout) 
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 0 (Timeout)
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 0 (Timeout)
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 0 (Timeout)
poll([{fd=13, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)

Now we know to look for fd13

ll /proc/47029/fd | grep 13
lrwx------ 1 root     root     64 Jul 20 10:20 13 -> socket:[1820320299]

Now we need to find out what socket that is (lsof it!!)

lsof -i -a -p 47029 | grep 1820320299

Sample output:

php-fpm 47029 nelsonweb   13u  IPv4 1820320299      0t0  TCP briansnelson.com:37253->255.255.255.255-static.hfc.comcastbusiness.net:7057 (ESTABLISHED)

Now you know where the connection is timing out, this also works for finding out if its mysql/memcache/redis sockets.