First, some context. I was finding that the box that houses this weblog and other stuff was locking up, for most services, in a matter of hours. There was a slow but steady decay in usability and I had no idea what it was. There were cryptic messages in the logs about processes using up all the file descriptors on the system, directing me to the tuning man page.
I finally figured it out last night but not before I had one additional spell of catatonia on my system.
I used mrtg to graph the increase in file descriptor usage and it made a nice straight line at about 45°, not a curve, no movement that would be consistent with things happening on the system. That looked suspicious.
I also set up a simple while loop in my shell to keep an eye on this. (while [ 1 ] ; do sysctl kern.openfiles; sleep 60; done ). When that stopped updating at about 6600 files (the max is set at 16384 or so), I rebooted (console was dead).
In the process of looking at this, I had noticed that there were a few nmbd (samba) processes in the process table: I expect to see one or two, but not 10 or more. I watched this after rebooting, and sure enough, new processes were spawning ever couple of minutes, and this with no logins to the samba shares.
I killed the samba processes, deinstalled samba, refreshed from cvs and, noting that the version was the same (2.2.8a), I reinstalled with portinstall -P.
That seemed to do it. Now openfiles are sitting in the low 200s . . . .