OSNZ

While I wont go into detail here, as there are plenty of “getting started with Salt” sites out there, I thought I’d mention that I’ve been using Salt quite a bit in the last few months for a couple of different customers and it’s been brilliant.

Sure, it has its foibles, but what doesn’t. It’s a brilliant tool for managing and maintaining groups of servers.

Notice I say groups of servers. If you don’t have at least 4 hosts which need similar configuration, I wouldn’t bother, the overhead wouldn’t be worth it. But for 4 or more it’s definitely worth the effort. Even for just setting up the “common” elements of a host like SSH keys, base package sets, etc.

Beware though, you need to keep your Salt master SECURE. It holds the keys to the kingdom, having root access to all your minion host. Although I understand it is possible to set up the minions as a non-root user, this is unusual and I understand it’s not straight forward.

Salt is very similar to Puppet (and a few other host management tools) but is Free and very versatile. It can manage Linux and Windows hosts, although the available former is more fully featured/supported.

Python Django provides shell access for debugging, and that shell has tab completion which is _very_ useful. I wanted to enable the same functionality in the standard Python shell.

This post tells us how:
http://stackoverflow.com/questions/246725/how-do-i-add-tab-completion-to-the-python-shell

TLDR:

Create a file .pythonrc

# ~/.pythonrc
# enable syntax completion
try:
    import readline
except ImportError:
    print("Module readline not available.")
else:
    import rlcompleter
    readline.parse_and_bind("tab: complete")

then in your .bashrc file, add

export PYTHONSTARTUP=~/.pythonrc

I’ve been using Nagios and Cacti for many years, but they have their annoyances, not least the fact that they’re independent so require configuration in 2 places.

So I’ve been moving over to using Zabbix. Don’t get me wrong, Zabbix can be a pain too, but it does alerting and graphing in one app, the configuration is sane, and the flexibility around alerts blows Nagios away.

So I had some rather complex and beautiful graphing of Bind statistics in my old Cacti and I wanted to reproduce it in Zabbix. I couldn’t find anything close to what I wanted, so I rolled up my sleeves and created it from scratch.

The result is a script, a config and a template which collects almost every statistic available from Bind via its HTTP/XML statistics, some autodiscovery of per-zone statistics and a bunch of pretty graphs.

The code and setup instructions can be found on the GitHub page:

https://github.com/Pesticles/Zabbix-Bind9-Statistics-Collection

Please feel free to clone it and submit changes back to me.

I’ve had some issues lately with Ubuntu and Intel NICs, specifically with VLANs. I’ve also heard anecdotal evidence that there are issues with the driver version shipped with the stock kernels and that the latest version available from Intel fixes the issue.

So I’ve downloaded the updated driver from here:

https://downloadcenter.intel.com/SearchResult.aspx?lang=eng&ProdId=2255

At the time this was written, the latest version was 3.1.0.2

It installs fine on my test Ubuntu 12.04 machine, so now I want to ensure it gets installed automatically whenever a new kernel gets installed (approximately every few months). To do this I enlist the help of DKMS, or Dynamic Kernel Module Support system.

You’ll be compiling the module so you’ll need to install “build-essential” and your kernel headers, for the stock kernel this is “linux-headers-server”. You’ll also need the “dkms” package.

Download the driver tarball from the intel website (above) and untar it into /usr/src. Mine untarred to /usr/src/e1000e-3.1.0.2/

Create the dkms.conf file in this directory as follows: (Note: Depending on your browser, you may need to correct the double quote characters “ after copying-and-pasting)

MAKE="make -C src/ BUILD_KERNEL=${kernelver}”
CLEAN=“make -C src/ clean”
BUILT_MODULE_NAME=e1000e
BUILT_MODULE_LOCATION=src/
DEST_MODULE_LOCATION=/kernel/drivers/net/ethernet/intel/
PACKAGE_NAME=e1000e
PACKAGE_VERSION=3.1.0.2
AUTOINSTALL=yes
REMAKE_INITRD=yes

Add it to DKMS:

dkms add -m e1000e -v 3.1.0.2

Test build:

dkms build -m e1000e -v 3.1.0.2

And install:

dkms install -m e1000e -v 3.1.0.2

The above build and install will do so for the currently running kernel, if you want to do so for a specific kernel, add “-k kernel-ver” where kernel-ver is the kernel version, for example “-k 3.2.0-70-generic”.

And you’re done. The module will automatically compile for each new kernel that gets installed.

Luke.

Recently I’ve been playing around with Zabbix, as a potential replacement for my existing Nagios+Cacti monitoring system. As part of this I’ve decided to instrument all own equipment through Zabbix, to get the hang of it.

The zabbix packages available for Debian Stable/Wheezy are somewhat out of date, v1.8 and although v2.2 is available through the backports repository I decided I may as well get them straight from the source, as Zabbix have a Debian repository of their own with the latest packages.

This is fine for i386 and x86_64 architectures, but they do not provide packages for the armhf architecture that Raspbian uses, nor is there a backports repo available (that I could find)

v1.8 is OK, but I wanted some specific features from 2.2 in my agent, so I decided I might as well compile them myself from source, and once I had, it made sense to share them in case anyone else had the same need.

They are available from the packages.osnz.co.nz repository, you can easily install the repo and it’s public key by downloading and installing:

http://packages.osnz.co.nz/debian/osnz-release-wheezy.deb

Once that’s done, you can install any of the zabbix packages by running “apt-get install zabbix-packagename”

I plan to keep this repo up to date as the upstream zabbix repo releases source files.

Luke.

I think docopt should be getting more attention than it is. If you’re a programer and have _ever_ lived the nightmare that is command line argument parsing, you’ll like this library.

Luke.

Whenever I configure a VLAN on a Linux host interface, I like to name it. “dmz” is a much better name for an interface than “eth0.11” when you’re working with routes, or iptables, etc.

In the past I’ve used a fairly simple configuration stanza in /etc/network/interfaces to achieve this:

auto dmz
iface dmz inet manual
  up ip link set eth0 up
  up vconfig add eth0 11
  up ip link set eth0.11 name dmz
  up ip addr add 1.2.3.4/24 dev dmz
  up ip link set dmz up
  down vconfig rem dmz

However recent changes in the ifupdown package, and in the kernel, have meant I’ve had to expand and modify this stanza somewhat, here’s the new version for 14.04 (and 12.04 since the ubuntu11 version of the ifupdown package):

auto dmz
iface dmz inet manual
  pre-up ip link set eth0 up
  pre-up vconfig add eth0 11
  pre-up ip link set eth0.11 name dmz
  up ip addr add 1.2.3.4/24 dev dmz
  up ip link set dmz up
  post-down ip link set dmz name eth0.11
  post-down vconfig rem eth0.11

Here’s why.

In the ubuntu11 patch of the ifupdown package they added a directive for manual interfaces that gets automatically run on up:

ip link set dev %iface% [[mtu %mtu%]] [[address %hwaddress%]] up

This is so you can set the mtu and/or mac address using configuration directives (although I don’t see why you wouldn’t just do it manually if you wanted to, it’s a manual interface after all) however in the case of my original config, the interface named “dmz” doesn’t exist yet since our “up” lines are run _after_ the systems up line above.

So we need to instantiate the vlan and rename it in pre-up.

Finally, there seems to be a bug in sysfs (and/or the kernel) where an interface is renamed then removed. I don’t know if this is specific to VLANs, but it caused a couple of kernel Oopses and broken network stacks before I figured out what was going on.

When you rename a vlan, a symlink is created in /sys which maps the original name to the new one. If you “vconfig rem dmz” that symlink (which would normally be deleted) is left hanging, or pointing to a non-existent file. This appears to upset the kernel quite a bit, but only if the interface is removed, then you attempt to add it again (ifdown, ifup)

The answer is to rename the interface back to its original name before removing the VLAN. This ensures the symlink is removed properly and the interface can be downed and upped and downed repeatedly without problems.

Luke.

Highpoint sell a line of low-cost disk and RAID controllers with Linux drivers and utilities. While the price point of these devices is excellent, you get what you pay for to an extent; reports of their tech-support are mixed at best (I’ve not experienced it myself) and the quality of their install documentation and packages aren’t great.

I recently purchased a 640L RAID controller and spent more than 2 days getting it working with the current Debian stable. After much wailing and gnashing of teeth I’ve ironed out all the kinks and the result is not too bad. Write speeds >300MB/s and read speeds of around 200MB/s across 3 Western Digital “Red” 1TB HDDs in a RAID 5 configuration. That’s is about as good as you could reasonably expect from this arrangement.

So that you don’t have to experience the same frustration, and because I couldn’t find any similar documentation online when I was looking, here is a step-by-step to getting the driver and utilities installed on Debian “Wheezy”

Keep reading

At OSDC2013 I heard about a new open source CMS framework, created here in New Zealand, called SilverStripe.

I’ve had a chance to check it out and it looks like a nice little number. It installed easily on my Debian VM and I was up and running with a demo site in a few minutes. The tutorials lead you through some of the basic features available without diving into too much PHP, and it looks great.

It uses a basic templating language, which should be easy for anyone who understands HTML and has any one of the multitude of other templating languages out there.

All in all very nice for rapid deployment of a basic site. It remains to be seen how nice it is when you really need to get into the guts to do things “outside the ordinary” but if the polish on display so far is any indication I’m prepared to be pleasantly surprised.

If you’re looking for a PHP/MySQL based CMS that’s simple and easy to use, take a look at SilverStripe.

Luke.

OSDC has begun, currently listening to an excellent keynote by the CEO of silver stripe, a CMS system that I will definitely be checking out soon.

Day 2 of Kiwi Pycon, the NZ Python User Group annual conference, started with an excellent keynote by Jessica McKellar appealing for help from the community to improve the Python install experience for Windows users, amongst other things.

Yesterday saw a keynote by Michael Foord on the status of Python, as well as talks on Metaclasses, Autopilot, Maltego, NuPIC and much more.

Today promises talks on IPython Notebook, decorators amongst others.

This is my first Kiwi PyCon and I’m very impressed with the conference so far. I suggest if it comes to you neck of the woods next year, you take the time to attand, you wont regret it.

Luke.

Yesterday I came up against a problem I didn’t immediately have an answer to, something that doesn’t happen all that often. Today I found an answer and I thought I’d share it.

Keep reading

I was building an LDAP authentication host a few days ago and was setting up SSL to enable TLS support. One of the tricks with TLS is that because it runs over the same port as non-TLS traffic, it can be tricky to tell if it’s actually being used. TCPDump by itself won’t neccessarily be enough to be sure that your LDAP session is being encrypted.

To be certain I wanted to see the results of my configuration changes in Wireshark, but shipping dumps back from the server to my laptop each time I did a test was annoying. What I needed was live capture into Wireshark.

Here’s how you do it:

Looks simple, and it is once you get your head around it. Basically we’re using NetCat (nc) at each end of a SSH tunnel to get traffic from STDOUT of TCPDump to STDIN of Wireshark.

At the server end you want TCPDump to capture full size (actually this parts optional), raw packets (in this case TCP port 389 on eth0) and send them to STDOUT and pipe them to NC in listener mode (in this case, listening on port 31337):

tcpdump -s 1500 -w - -i eth0 tcp port 389 | nc -l -p 31337

Now we make a new SSH session from our local machine, with a tunnel configured through to that listener:

ssh user@remote-server -L 31337:localhost:31337

If you haven’t seen ’-L’ before, the format is LocalPort:RemoteHost:RemotePort. The RemoteHost address is as seen by the remote-server, in this case we want to connect to the listener we’ve just set up on remote-server itself, so we connect to “localhost”. The LocalPort specifies which local port on my local machine the tunnel is exposed as, in this case I’m using the same 31337 but it could be anything.

Finally we use NetCat again to connect to the tunnel and output the data that comes from it to STDOUT, which we pipe to Wireshark:

nc localhost 31337 | wireshark -k -i -

Note that the “localhost” this time is our local machine, we’re connecting to the port SSH has exposed. Wireshark is told to read its packet data from STDIN with “-i -”

And that’s that. Any packets that tcpdump captures will immediately appear in your local wireshark which makes diagnosing application layer issues a doddle.

Luke.

For anyone who doesn’t already know, the Raspberry Pi is a tiny computer based on a cellphone chip, which runs Linux (amongst other things) and costs less than $50. I’ve had 2 for a few months, bought out of interest and novelty, I didn’t really have a clear idea what I’d do with them.

Recently however, I’ve been working to turn one into a music player of sorts and the experience has been a rewarding one.

The OS I’ve chosen is Raspbian, a Debian variant provided by the Raspberry Pi creators and the community. It’s lightweight, provides a desktop if you need it (I don’t in this case) and has the full power of Debian and all its packages behind it.

One of those packages is the remarkable “mpd” or Music Player Daemon. This little gem is a jukebox in a box. Easily scriptable from the command line and with just about every feature you could want (it’s got everything I’ve needed so far, including shuffle, repeat, playlists, volume controls and even track crossfading!)

Pair that with Django to whip up a quick web-based interface and viola! you’ve got a tidy little player.

Admittedly there was some wrangling required to get the sound right, the default drivers are renowned for loud pops between tracks as the audio device is turned off and on, but there are several workarounds posted online and I found one that worked for me. Now the sound quality is perfectly acceptable.

Todays trick was getting a USB WiFi adapter working and I was a little dubious since the last time I tried to get wireless working without a GUI a few years ago was fraught with issues, wrangling wpa_supplicant and iwconfig. Seems things have come along though and only 2 configuration lines in /etc/network/interfaces to define the SSID and passphrase and POW! Connected.

This little thing continues to surprise and delight me.

Another possible project I have in mind is a web terminal/kiosk for my sisters business. She has a new online timekeeping system and she wants web terminals near the equipment but was concerned at the cost of buying several PC’s, even super cheap ones. I pointed out that a small screen, plus RPi, plus mouse and keyboard would be under $200, she seemed to like the idea :)

If you’re a tinkerer, I recommend you try one of these out. It’d be a bargain at twice the price.

Luke.

I’m nearing completion of stage 3 of my home instrumentation project, and thought it was about time I mentioned it here.

The intention is to be able to tell, from anywhere at any time, the vital statistics of our house, and to be able to look back and see what’s happened in the past.

I’ll be posting some of the code and some how-to type docs in future but for now a description will do.

Keep reading

And a Happy New Year to all!

I’ll be hanging around Auckland over the break but will be seeing plenty of family and friends, I hope you do the same.

Don’t drink too much, see you on the other side :D

Luke.

New Zealands best security oriented conference is on again!

On November 17th and 18th Wellington will once again play host to bogans, black hats and the fuzz in a 2 day orgy of hacking, cracking and 0-day. Details are scarce so far, it’s early days yet, but I’ve no doubt it will be another eye opening experience.

This will be my third Kiwicon. Each time it serves to refresh my paranoia, reinforce my strongly held opinions on the security (or lack of it) of the systems we use every day and strengthen my tendency to be bullish about scurity concerns when designing and implementing my own systems.

At Kiwicon 2010 I witnessed the power of one, when a lone hacker outpaced groups of 5 or more in the hacking competition, perched on the arm of someone elses chair with a tiny netbook on his knee.

Kiwicon 2011 scared me off Apple i-devices when it was demonstrated just how easily their “security” could be bypassed, even on the latest version (at the time) of iOS.

Every time Brett Moore speaks I’m amazed by the depth and breadth of the attack surface available to the dark side. Every attempted mitigation made by software and hardware vendors seems to be compromised by, well, compromise. The need to make something backwards compatible is often the downfall of an otherwise elegant and robust solution. Heap randomisation you say? Wow, that would make buffer overflows really hard to exploit, but what’s that? I can turn it off for backwards compatibility? Awwww.

No doubt this year will bring more eye opening tidbits, I look forward to devouring every one. Also the beer.

Hope to see you there!

www.kiwicon.org

Or, in fact, the Source.

I’m no artist, but I do love to create and my chosen medium is code. Being fairly unimaginative I usually rely on real-world requirements to inspire my projects; fixups, glue and gap-fillers have been my calling for my entire career as an engineer.

Over the years I’ve written quite a few bits of code and some deserve to see the light of day. Besides, it’d be rude to call myself an Open Source expert and not release the source to my own work.

To that end I plan to introduce a new section to the OSNZ website, publishing as many of my little (and not-so-little) code snippets as is practical. Hopefully someone will find them useful.

Watch this space.

Luke.

I’ve come across some really curly mail challenges in my time. The customer wants mail delivered to a user, specified not in the To: field, but in the

subject line you say? Oh, you want to deliver mail to users authenticated from Active Directory, but there are 3 domains and users might exist in any (and more than) one?

Both of these are problems I’ve encountered in real life, amongst many others, and every time I’m able to lean on Exim, knowing there will be a way.

I am in awe of the power and flexibility of this mailer. There is literally nothing it can’t do. It can do Lookups against almost anything you can think of and can integrate easily with content filtering for antivirus and antispam. You can even extend it with Perl, if you want to get funky.

And what’s even better is that the configuration is so easy to use and understand. Everything is in there, not just changes to defaults as is the case with most software. It’s more like a delivery instruction manual than a configuration. This might scare people off initially, it’s a big file, but the beauty of it is you can see exactly what’s going to happen at every stage of a messages delivery cycle. And change it if you want to.

It helps of course that Exim on Debian is extremely well packaged. The configs are well commented and easy to extend. However Exim can be just as easy and powerful on other platforms, with a little extra effort.

If you’re still struglling away with Postfix or, heaven forbid, Sendmail, join the 21st century and try out Exim. You won’t regret it.

Luke.

I’m presently performing my first customer migration to Google Apps, primarily to replace an exchange server. 9 users migrating mail, calendars and contacts but wanting to continue using Outlook to smooth the transition.

Google Apps Sync makes this a breeze, and (mostly) transparent to the user. This tool continues to improve, making Google Mail access seamless through the Outlook client.

While there are some minor annoying niggles in the administration web interface for Apps, they will be corrected in time I’m sure. One of the advantages of cloud services is product improvement without having to install updates.

I believe this option should be considered by any small business not wanting to administer their own server, but wanting more features than their ISP offers.

Luke.