NTP attack

Yep we got bit by this one. It brought our network to its knees. Fortunately the fix was easy. First, to stop the attack we turned off ntp temporarily:

service ntp stop

Then we added this to our /etc/ntp.conf file:

Disable monitor
Restrict -4 default kod notrap nomodify nopeer noquery

Then we started ntp:

service ntp start

The system returned to normal.

FreeBSD update broke Apache perl modules

After a recent freebsd update fetch && freebsd update install Apache would not restart properly. It was complaining about missing perl modules it relied on. So we rebuilt the perl port and all its dependencies.

sudo portmaster -m BATCH=yes --no-confirm -D -r perl

The -m BATCH=yes chooses defaults at setup screens and bypasses them, --no-confirm avoids prompting at the command line, -D keeps distfiles in tact and -r updates all its port dependencies. Once this was done Apache fired right up!

ZFS zpool crash

Recently we had an issue where one of our servers unexpectedly crashed and we had to hard reboot it. When it rebooted we began seeing kernel panics and it would not complete its boot cycle. In comes my mfsbsd stick to the rescue! The machine in question was running FreeBSD 10. We had just built it so we had one of these sticks handy. To create a mfsbsd stick yourself grab the image here:
http://mfsbsd.vx.sk

FreeBSD Build Image Instructions:
http://www.freebsd.org/doc/en/articles/remote-install/preparation.html

We ended up using Win32 Disk Imager:
https://sourceforge.net/projects/win32diskimager/

Once we got booted off the usb stick to a command prompt (beyond the scope of this post but simple, follow the prompts) we simply did the following to remount the zfs pools that were fubar’d:
zpool import -f -a

All zfs pools imported with no problems. Then we rebooted the machine as normal and voila, no errors!

Windows 8 Power User Menu

Bring up the power user menu in Windows 8:
WINKEY-X

A life saver when you are a UNIX admin/user trying to setup a Windows 8 machine.

Some of the more useful admin shortcuts displayed are:
Event Viewer
System
Device Management
Disk Management
Command Prompt
Task Manager
Control Panel
Run
Desktop

Fun with a FreeBSD named port upgrade.

::Begin Prologue::
Recently we performed the following on our DNS server running FreeBSD 9.1:
freebsd update fetch && freebsd update install
Always smooth sailing. 🙂
::End Prologue::

So, today we got a new laptop which meant setting up the DNS records for it to run on our network. We added the necessary forward and reverse DNS records in named. After doing so, the normal practice is to run rndc reload so the system will reload and recognize the new DNS records. A very simple process. But today was a very different story. Today, running rndc reload produced this error message:

rndc: neither /usr/local/etc/rndc.conf nor /usr/local/etc/rndc.key was found

Huh? Out of the blue rndc does this? Well guess what, the named port got upgraded, so go figure.

Now we noticed the symbolic link /etc/named/rndc.key to /usr/local/etc/rndc.key was missing, so let’s create it:
ln -s /etc/named.rndc.key /usr/local/etc/rndc.key

Now running rndc reload produced this error:

rndc: 'reload' failed: not found

It would have been nice if the message had told me exactly what file was not found. But looking in the /var/log/daemon.log file pointed me in the right direction as to what was going wrong:

May 19 15:11:19 ns named[61129]: received control channel command 'reload'
May 19 15:11:19 ns named[61129]: loading configuration from '/usr/local/etc/named.conf'
May 19 15:11:19 ns named[61129]: open: /usr/local/etc/named.conf: file not found
May 19 15:11:19 ns named[61129]: reloading configuration failed: file not found

Normally it should look in /etc/named/named.conf for the configuration file but our new version now has a new path it is looking in.

So we then added the following to the /etc/rc.conf file:

named_conf="/etc/named/named.conf"

And ran:

service named restart
Stopping named.
Starting named.
/etc/rc.d/named: WARNING: failed to start named

Blamo! Named stopped and could not start back up, no thanks to our new config line. It again produced the same error in the log file. And now, there was no named service running. Double ugh.

This command fixed that and pointed it to the correct config file:

/usr/local/sbin/named -t /var/named -u bind -c /etc/namedb/named.conf

Now the named service is up and running and rndc reload runs as normal. Great!
Now we just need to do a reboot of the system (when there is little traffic) to see if named starts up normally after adding that line to /etc/rc.conf.