linux.conf.au: day 2

The second day of linux.conf.au has been and gone, was another day of interesting miniconf talks and many geeky discussions with old and new friends.

Jethro: Booted

Jethro: Booted, with the power of coffee!

The keynote was a really good talk by Radia Perlman about how engineers approach developing network protocols and an interesting talk of the history of STP and the designed replacement, TRILL. Great to see a really technical female keynote speaker at LCA this year, particularly one as passionate about her topic as Radia.

The conference WiFi is still pretty unhappy this year, I’ve been suffering pretty bad latency and packet loss (30-50%) most of the past few days – if I’ve been able to find an AP – seems they’re only located around the lecture rooms. Yesterday afternoon it seems to have started improving however, so it may be that the networking team have beaten the university APs into submission.

No internet makes sad Jethro sad. :'(

No internet makes sad Jethro sad. :'(

Of course, some of the projectors decided not to play nicely, which seems pretty much business as usual when it comes to projectors and functioning…. it appears that the projector in question would complain about the higher refresh rates provided by DVI and HDMI connected devices, but functioned correctly with VGA.

Someone did an interesting talk a couple of LCA’s ago on the issue, apparently many projectors lie about what their true capabilities are and request resolutions and refresh rates from the computer that are higher than what they can actually support, which really messes with any modern operating system’s auto-detection.

Lending my VGA enabled Thinkpad to @lgnome whist a @chrisjrn observes.

Lending my VGA enabled Thinkpad to @lgnome whist a @chrisjrn observes.

A startled @colmiga approaches!

A startled @colmiga approaches!

Geeks listening intently

Geeks listening intently to concurrent programming.

@lgnome pushing some crazy new drugs to all the kiddies

@lgnome pushing some crazy new drugs to all the kiddies

A few of my friends were delivering talks today, so I spent my time between the Browser miniconf and Open Programming miniconf, picked up some interesting new technologies and techniques to look at:

  • Adam Harvey’s PHP talks were great as usual, always good to get an update on the latest developments in the PHP world.
  • Francois Marier from Mozilla NZ presented on Content Security Policy, a technique I wasn’t aware of until now. Essentially it allows you to set a header defining which sites should be trusted as sources of CSS, Javascript and image content, allowing a well developed site to be locked down to prevent many forms of XSS (cross site scripting).
  • Francios also spoke briefly about HTTP Strict Transport Security, a header which can be used by SSL websites to fix the long standing problem of users being intercepted by a bad proxy and served up a hacked HTTP-only version of the website. Essentially this header tells your browser that your site should only ever be accessed by HTTPS – anything that then directs your browser to HTTP will result in a security block, protecting the user, since your browser has been told that the site should only ever be SSL from it’s previous interaction. It’s not perfect, but it’s a great step forwards, as long as the first connection is made on a trusted non-intercepted link, it makes man-in-the-middle attacks impossible.
  • Daniel Nadasi from Google presented on AngularJS, a modern Javascript framework suitable for building complex applications with features designed to reduce the complexity of developing the required Javascript.

After that, dinner at one of the (many!) Asian restaurants in the area, followed by some delicious beer at the Wig and Pen.

Either I've already had too many beers, or there's a giant stone parcel in my way.

Either I’ve already had too many beers, or there’s a giant stone parcel in my way.

Onwards to delicious geekiness!

Onwards to delicious geekiness!

Delicious hand pulled pale ale.

Delicious hand pulled pale ale.

The beetroot beer is an interesting idea. But some ideas should just not be attempted. :-/

The beetroot beer is an interesting idea. But some ideas should just not be attempted. :-/

Native Australian night life!

Native Australian night life! This little fellow was very up close and friendly.

Linux.conf.au native wildlife. ;-)

Linux.conf.au native wildlife. ;-)

Another great day, looking forwards to Wednesday and the rest of the week. :-)

linux.conf.au: day 1

First proper day of linux.conf.au today, starting with breakfast and the quest of several hundred geeks to find and consume coffee.

Some of us went a bit overboard to get their exact daily coffee fix....

Some of us went a bit overboard to get their exact daily coffee fix….

After acquiring coffee, we started the day with a keynote by the well known Bdale Garbee, talking about a number of (somewhat controversial) thoughts and reflections on Linux and the open source ecosystem in regards to the uptake by commercial companies.

Keynote venue.

Keynote venue.

Bdale raised some really good points, particularly how GNU/Linux isn’t a sellable idea to OEM vendors on cost – many vendors pay nothing for Microsoft licensing, or even make a profit due to the amount of preloaded crapware they ship with the computers. Vendors are unlikely to ship GNU/Linux unless there is sufficient consumer demand or feature set that makes it so good

My take on the talk was that Bdale was advocating that we aren’t going to win the desktop with a mass popularity – instead of trying to build a desktop for the average joe, we should build desktops that meet our own needs as power uses

It’s an interesting approach – some of the more recent endeavours with desktop developers has lead to environments that newer users like, but power users hate (eg GNOME 3), as a power user, I share this view, I’d rather we develop a really good power user OS, rather than an OS designed for the simplest user. Having said that, the nice thing about open source is that developers can target different audiences and share each other’s work.

Bdale goes on to state that the year of the Linux desktop isn’t relevant, it’s something we’re probably never going to win – but we have won the year of Linux on the mobile, which is going to replace conventional workstations more and more for the average use and become the dominant device used.

It’s something I personally believe as well, I already have some friends who *only* own a phone or tablet, instead of a desktop or tablet, and use it for all their communications. In this space, Android/Linux is selling extremely well.

And although it’s not a conventional GNU/Linux space we know and love and it still has it’s share of problems, a future where Android/Linux is the dominate device OS is much more promising than the current Windows/MacOS duopoly.

The rest of the day had a mix of miniconf talks – there wasn’t anything particularly special for me, but there were some good highlights during the day:

  • Sherri Cabral did a great talk on what it means to be a senior sysadmin, stating that a proper senior sysadmin knows how to solve problems by experience ( not guess work), works to continuously automate themselves out of a job with better tools and works to impart knowledge onto others.
  • Andrew Bartlett did a brief update on Samba 4 (the Linux CIFS/SMB file system implementation) – it’s production ready now and includes proper active directory support. The trade off, is that in order to implement AD, you can’t use an external LDAP directory or Kerberos server when using Samba 4 in an AD server mode.
  • Nick Clifford did an entertaining presentation on the experiences and suffering from working with SNMP, turns out that both vendor and open source SNMP implementations are generally quite poor quality.
  • Several interesting debates over the issues with our current monitoring systems (Nagios, Icinga, Munin, etc) and how we can fix them and scale better – no clear “this is the solution” responses, but some good food for thought.

Overall it was a good first day, followed up by some casual drinks and chats with friends – thankfully we even managed to find an open liquor store in Canberra on a public holiday.

Poor @lgnome expresses his pain at yet another closed liquor store before we located an open location.

Poor @lgnome expresses his pain at yet another closed liquor store.

 

 

linux.conf.au: day 0

It’s time for the most important week of the year – linux.conf.au – which is being held in Canberra this year. I’m actually going to try and blog each day this year, unlike last year which still has all my photos in the “too be be blogged folder”. :-)

Ended up taking the bus down from Sydney to Canberra – at only around $60 and a 3 hour trip, it made more sense to take the bus down, rather than go through the hassle of getting to and from the airports and all the security hassles of flying.

Ended up having several other linux.conf.au friends on the bus, which makes for an interesting trip – and having a bus with WiFi and power was certainly handy.

I am geek, hear me roar!

I am geek, hear me roar!

Horrifying wail of the Aucklander!

Horrifying wail of the Aucklander!

The road trip down to Canberra wasn’t particularly scenic, most of the route is just dry Australian bush and motorways, generally it seems between city road trips in AU tend not to be wildly scenic unlike most of the ones I take in NZ.

Canberra itself is interesting, my initial thoughts on entering the city was that it’s kind of a cross between Rotorua and post-quake Christchurch – most of the city is low rise- 5-10 story buildings and low density sprawl, and extremely quiet with both the university and parliament on leave. In fact many have already commented it would be a great place to film a zombie movie simply due to it’s eerily deserted nature.

Considering it’s  a designed city, I do wonder why they choose such a sprawled design, IMHO it would have been way better to have a very small high density tower CBD which would be easily walk-able and massive park lands around them. Canberra also made the mistake of not putting in light rail, instead relying on buses and cars as primary transport.

Neat fountain in town

Neat fountain in town

The Aussies can never make fun of us Kiwis and sheep again... at least we don't have THIS in our capital city O_o

The Aussies can never make fun of us Kiwis and sheep again… at least we don’t have THIS in our capital city O_o

Impressively large transmission tower for such a small city.

Impressively large transmission tower for such a small city.

Once nice side of Canberra, is that with the sprawl, there tends to be a lot of greenery (or what passes for greenery in the aussie heat!) around the town and campus, including a bit of wildlife – so far I’ve seen rabbits, cockatoos, and lizards, which makes a nice change from Sydney’s wildlife viewing of giant rats running over concrete pavements.

Sqwark!

Sqwark!

The evening was spent tracking down the best pub options near by, and we were fortunate enough to discover the Wig and Pen, a local British-style brewery/pub, with about 10 of their own beers on hand pulled taps. I’m told that when the conference was here in Canberra in 2005, the attendees drank the pub dry – twice. Hopefully they have more beer on stock this year.

First beer casualty from the conference - laptop being stood vertically to drain, whilst charging a cellphone.

First beer casualty from the conference – laptop being stood vertically to drain, whilst charging a cellphone.

Normally every year the conference provides a swag bag, typically the bag is pretty good and there’s usually a few good bits in there, as well as spammy items like brochures, branded cheap gadgets (USB speakers, reading lights, etc).

This year they’ve cut down hugely on the swag volume, my bag simply had some bathroom supplies (yes, that means there’s no excuse for the geeks to wash this week), a water bottle, some sunblock and the conference t-shirt. I’m a huge fan of this reduction in waste and hope that other conferences continue on with this theme.

Arrrrrr there be some swag me mateys!

Arrrrrr there be some swag me mateys!

The conference accommodation isn’t the best this year – it’s clean and functional, but I’m really not a huge fan of the older shared dorm styles with communal bathroom facilities, particularly the showers with their coffin-style claustrophobic feel.

The plus side of course, is that the accommodation is always cheap and your evenings are filled with awesome conversations and chats with other geeks.

Looking forwards for the actuals talks, going to be lots of interesting cloud and mobile talks this year, as well as the usual kernel, programming and sysadmin streams. :-)

Presenting NamedManager

A while ago I had a project to build a DNS management application for a client, which has since been refined and improved further, and finally released as “NamedManager” now that I’ve had time to re-do the documentation for a public audience.

NamedManager is an AGPL web-based DNS management system designed to make the adding, adjusting and removal of zones/records easy and reliable by a simple yet effective interface.

Rather than attempting to develop a new name server, NamedManager supports the tried and tested Bind name server and can integrate nicely into existing complex Bind configurations including servers with multiple views without clobbering custom configurations.

Configuring zone records with NamedManager.

Configuring zone records with NamedManager.

It’s written in PHP 5 and uses a MySQL database for storing the DNS record information, which is then converted into Bind compatible configuration files and copied to the name servers, an act which ensures that any loss of the NamedManager application or database will not result in a loss of DNS services.

It’s a stable application, having been in some large production environments for over a year, although there’s certainly more work wanted on the application, such as the addition of IPv6 PTR records and improved UI around SRV and SPF record entry.

NamedManager includes an interface for tracking the sync status of the latest changes across all your name servers, as well as understanding the differences between internal only and publicly accessible name servers and generating the appropriate NS records for domains automatically.

An included daemon can (optional) watch the Bind name server logs and send them back to the web interface, so that you can watch all your name servers via an AJAX log interface to make it easier to watch for issues or debug queries.

Server status report - see if your hosts have synced DNS changes and are reporting logs.

Server status report – see if your hosts have synced DNS changes and are reporting logs.

Both forwards domains for IPv4 and IPv6 are supported, and IPv4 reverse domains are also supported (IPv6 reverse to come in future release), along with the ability to import Bind zone files (works for most, unless yours is too ugly/complex) into the application.

View of all the domains active on this DNS cluster with NamedManager.

View of all the domains active on this DNS cluster with NamedManager.

For developers, NamedManager features a SOAP API which can be used to manage DNS records – this has been used to hook into other provisioning tools (eg: cloud instance management tools) to reduce manual effort for keeping records clean and tidy,

The code structure of NamedManager would make it possible to add support for additional name servers as desired, I would be keen to see support for PowerDNS and Amazon Route 53 as options in future – as always, patches welcome. ;-)

If you’re interested in checking it out, view the NamedManager project page here and follow the instructions to install from RPM, source tarball or SVN.

Pictures from the Sydney waterfront

This post is a collection of various pictures from around Sydney Harbour that I’ve been collecting as I’ve been here.In my opinion, the waterfront is the best feature of Sydney –  the city isn’t wildly exciting to me, it’s a somewhat generic large city, however the waterfront adds some uniqueness and difference to it and there’s always something different taking place on the water.

Sydney CBD and bridge in the hazy distance.

Sydney CBD and bridge in the hazy distance.

A water ferry cruises through Darling Harbour, near the maritime museum.

A water ferry cruises through Darling Harbour, near the maritime museum.

View out from Pyrmont towards the Harbour Bridge.

View out from Pyrmont towards the Harbour Bridge.

They put everything on barges here... I've seen party barges, crane barges, cement mixer barges even a soccer field barge (!!).

They put everything on barges here… I’ve seen party barges, crane barges, cement mixer barges, a soccer field barge and even a barge full of strippers and sleazy bankers. :-/

Sydney's military port - I really want to find out what they use that massively bulky crane to lift - tanks maybe?

Royal Australian Navy’s primary dock – I really want to find out what they use that massively bulky crane to lift – tanks maybe?

One of the smaller ferries comes in to dock at Circular Quay.

One of the smaller ferries comes in to dock at Circular Quay.

Pulling away from Circular Quay on the Manly Ferry

Pulling away from Circular Quay on the Manly Ferry

All hail duck kind!

Bow before Lord Carr and his mighty duck army!

Mysterious discharge from a mysterious pipe slung under a pier.

Mysterious discharge from a mysterious pipe slung under a pier.

Looking out towards Pyrmont as the sun sets.

Looking out towards Pyrmont as the sun sets.

Sydney CBD to Manly

Since living in Sydney, I’ve been out to Manly a couple times, including for a walk around the North Head trails which showcase some of Australia’s military history and fortifications.

Manly has a lot going for it, offering great beaches, walking trails and a strong ecosystem of different shops and food venues. And with easy access from Circular Quay via a scenic 30minute ferry ride, it’s a great day outing for me, especially since Circular Quay is only a 20min walk from my flat.

One thing I have noticed on my ferry trips out to Manly, is that there’s a lot of scenic bush and bays all along the North Sydney coast time from the Sydney Harbour Bridge right out to Manly. I therefore decided to make Sydney CBD to Manly my next walking adventure in Sydney.

Of course, those readers familiar with Sydney will say – “but Jethro, that’s nuts, Manly is miles away!”.  Well dear reader, I can confirm for you – Manly is a painstaking 30.04km pain-inducing death-march from Sydney CBD, and I even have the GPS trail for you to follow if you wish to re-create this pain in your own life. :-)


View Walk from Sydney CBD to Spit Point Bridge in a larger map

It’s an extremely interesting walk, along the way you pass through opulent suburbs, the back of Sydney Zoo, numerous coves and beaches, a number of hills, bush, WW2 fortifications and two major bridges.

Please excuse the line breakage on the map – Google Track didn’t handle some GPS plot absences particular well, I’ve touched up as best as I can, but couldn’t figure out a way to connect segmented lines. There’s also the period from Spit Point Bridge to Manly itself where my phone battery died, so I’ve drawn in the lines to a mostly-accurate level, but it messes with the total distance stats reported and leaves some weird graphic artefacts.

It’s best viewed in the full Google Maps interface – I’ve tagged certain areas of interest, it’s also worth noting the location of Ferry docks, since there’s a number of good attractions that you may wish to ferry out directly to go and see.

I took a few photos along the way – the trip is very scenic, I really need to go back with a DSLR as I came across heaps of different things I would have loved to capture in a better level of quality.

Sadly Lisa didn’t want to join me on this walk for some strange reason, so you’ll have only pictures of pained geek and scenary, rather than any actual eye candy from my trip photos:

Full of optimism, I set forth from my apartment with my trust RedHat t-shirt to bring geek culture to the bush.

Full of optimism, I set forth from my apartment with my trust RedHat t-shirt to bring geek culture to the bush.

The first stage of the walk is through Sydney CBD and over the Sydney Harbour Bridge – I have plenty of photos from a previous walk across if you’re interested. The bridge is always an interesting stroll and only takes a couple minutes offering amazing views the whole time.

Once off the bridge in Milsons Point, the walk took me through the suburbs and coves directly around North Sydney. The area is certainly wealthy, but there’s odd pockets of housing or apartments that appear otherwise.

The whole route is littered with small coves and marinas like this one.

The whole route is littered with small coves and marinas like this one.

Looking out back towards Sydney

Looking out back towards Sydney

Amusingly odd to have man made items pushing their way through the path, rather than tree roots.

Amusingly odd to have man made items pushing their way through the path, rather than tree roots.

The walk through these suburbs is nice enough, but the real gold starts once the beach and coastline tracks start, these offer a much desired escape from the city noise and people and some amazing views of the harbour and Sydney itself.

City looks pretty far away already!

City looks pretty far away already!

The walkways that have been installed behind the Sydney Zoo are particularly good, these paths are well formed and often form wooden stairs and walkways for a good few KM of coastline.

Walking out behind Sydney Zoo towards the ferry dock.

Walking out behind Sydney Zoo towards the ferry dock.

Thankfully the day was overcast and only low 20s, otherwise I'd be looking a lot worse for wear.

Thankfully the day was overcast and only low 20s, otherwise I’d be looking a lot worse for wear.

A number of trees along the route with this weird base hollowing.

A number of trees along the route with this weird base hollowing.

Also a number of trees with this weird growth on them - seems to be too many for them to be beehives

Also a number of trees with this weird growth on them – seems to be too many for them to be beehives… dropbear nests?

Weird bark stripped trees - came around many of these, the trees have all their bark stripped off and lying around them - unsure as to why.

Weird bark stripped trees – came around many of these, the trees have all their bark stripped off and lying around them – unsure as to why.

Halfway between the Sydney Zoo and Chowder Bay where the well built Zoo trail ends, there’s Bradley Head, an area of parkland with a small lighthouse and the distinctive tripod mast from the original HMAS Sydney WW1 era ship.

Mast from the HMAS Sydney (1912) WW1 era ship at Bradley's Head.

Mast from the HMAS Sydney (1912) WW1 era ship at Bradley’s Head.

Following the coast around from Bradley’s Head and through Chowder Bay, leads to Middle Head, home of the Middle Head Fortifications that guarded Sydney Harbour. With direct line of sight to any ships attempting to pass through Sydney Harbour, it offered an ideal defensive location, featuring both cannon and disappearing guns – similar to the ones in Devonport NZ, however there’s no longer any guns located at Middle Head.

View out from Middle Head towards Sydney Heads, the entrance to Sydney Harbour.

View out from Middle Head towards Sydney Heads, the entrance to Sydney Harbour.

10 points if you spot the balloon tagged body.

Old fortifications: everyone loves lying around in them with a balloon tied to their body.

Unsure if this was a half-assed demolition job, or whether the building broke apart over the years.

Unsure if this was a half-assed demolition job, or whether the building broke apart over the years.

Exercised geek - at this point I was still feeling pretty good.

Geek, with exercise add-on. At this stage I was feeling a little tired, but still needed to head down to Hunters Bay, and then back up a big hill before I could then go down across Spit Point Bridge.

There’s quite a bit of stuff to explore out at Middle Head, it’s something that I’ll have to make another visit to in future to properly explore – there’s no quick ferry route to Middle Head, closest terminals appear to be either the zoo dock, which will require a few more KM of (pleasant) walking.

Walking down towards Spit Point Bridge. Just about every cove I see has a marina situated in it, must be almost as many boats as cars in Sydney.

Walking down towards Spit Bridge. Just about every cove I see has a marina situated in it, must be almost as many boats as cars in Sydney.

The walk along Spit Rd to the Spit Bridge isn’t particularly great, as it’s a main road (2x 2-lanes roads) the way down, so this section of walk has all the charms of walking alongside a motorway, however it’s the only way across (short of swimming).

Spit Point Bridge! I just managed to run across as the lights were flashing before it started lifting.

Spit Point Bridge! I just managed to run across as the lights were flashing before it started lifting.

I fucking love bridges!

I fucking love bridges!

Around this time my phone decided it didn’t want to handle more than 4 hours doing GPS track plotting and died, so the remainder of my walk to Manly has been drawn in manually on the Google Map above.

There is a coastal route the whole way from Spit Point Bridge to Manly, at about 10km. Having already done 22km by this stage and it getting late, I decided to cut back across some of the suburbs, rather than finding my way though the unfamiliar suburbs and do a shorter 7km trip.

In hindsight I should have continued up Spit Rd and onto Sydney Rd, taking the coast path part of the way made the walk back up to join Sydney Rd longer and harder than it needed to be.

Pain, oh god the pain.

Pain, oh god the pain. In some ways it was good that my phone died, so that pictures of me looking even worse couldn’t be taken.

I’m keen to do the proper Spit Bridge to Manly walk, so maybe I’ll make that segment a separate trip one day, or if I’m ever crazy enough, re-try the walk starting from Manly and walking back to the city.

I eventually made it into Manly, rapidly devouring an iced bun (mmm delicious 2000+ kJ energy) and a coke, the champion feed of any athlete (I was out of muesli bars ok!!) and got the ferry back to Circular Quay.

Being able to sit at the stern of the ferry and see all the coves I had walked around during the 30mins ferry journey back was actually incredibly relaxing, certainly a good ending to the trip. The only problem is that the half hour break in walking meant that my body started cooling down and it was not pleasant getting moving once back at Circular Quay.

I managed to strain one of my ankles slightly - the rest of me hurts like crazy, but in the usual exercise way, so it should come right in the next couple of days.

I managed to strain one of my ankles slightly – the rest of me hurts like crazy, but in the usual exercise way, so it should come right in the next couple of days.

General stats from the trip – I averaged 5.21km/h walking speed – my actual walking speed is more around 5.9km/h, I suspect time spent looking at sights and suburban traffic lights messed with those stats somewhat.

The total distance was 30.5km (some GPS measurement inaccuracies in that of course) – if I was able to keep that rate up daily, I would be able to walk from Wellington to Auckland in one month (of course reality is that there is a lot more time needed for breaks, detors, difficult terrain, etc).

I’d certainly recommend parts of this walk to anyone interested – however you have to be fucking nuts to do the whole thing like I did – I’m still hobbling around my apartment like an old man in pain.

Ebook Debate

Lisa and I recently decided to purchase a Kindle – having first moved to Auckland and then Sydney, we were unable to take many books with us, something we both regretted – as the plan isn’t to settle in Sydney long term, it’s not worth shipping them over to only then have to ship them elsewhere again.

In the last 5+ years almost all the reading I’ve done has been on my laptop, but this has some unfortunate side effects:

  • It’s never as easy to lie on the beach or at the park with a laptop as it is for with a book, as a result, we do far less reading or lying outside than we would like.
  • Years of working in IT has turned me into a screen skim reader – I was always a fast reader, but in order to keep up with analysing logs and complex information, I’m become a very impatient skim reader. If I try to read any large textual content such as books on my computer, I’m liable to skip through content.
  • Computers make me context-shift – I’ll be reading a book, and then jump into email, then IM and lose all depth and concentration on what I was reading. Sometimes I just want a book to be a book.
  • LCD screens are very difficult to read outside in bright sunlight – this is particularly an issue, since we want to try and get outside and out of our apartment more to escape the crippling heat of Sydney.

We pondered getting a small tablet, such as the iPad Mini or Google Nexus 7, but these share a lot of the same problems as using our laptops, such as the LCD screen being poor for reading and the multi tasking nature of the device leading to context shifting and loss of focus.

In the end, we decided to buy one of the current generation of basic Kindle models, and if we like it, upgrade to having two of the newer generation Paperwhite Kindles once they launch in the AU market.

We had a bit of a mission getting one – the Paperwhite is currently only available in the US market and Amazon won’t ship to AU – but when we tried to order the regular model via Amazon, it kept refusing to ship to AU for some unstated reason:

Hi Amazon, I have a bug report for you...

Hi Amazon, I have a bug report for you…

I ended up giving up and shelling out an extra $20 to buy from Dick Smith’s who had the model we wanted in stock. It’s been a long time since I shopped retail, it’s certainly an interesting experience….

With the expensive prices, display models that were two generations out of date, display Kindles that hadn’t even been setup leaving them unusable for demonstrations and pushy sales pitches trying to offload warranties, cases and screen protectors, it’s no wonder that online shopping is decimating the AU retail market.

Aside from the purchasing hassles, the Kindle is shaping up to be a great device – at least based on what I’ve experienced so far, which is only the basics, as Lisa keeps stealing it away from me….

The e-ink display is excellent in daylight, although it suffers a bit in our apartment, which has somewhat dull lighting levels – the newer Paperwhite model with the front-lit display would help resolve this issue for us, so I expect we will look to upgrade once it’s released.

In bright sunlight or even just outdoors in general, the display is clear and easy to read, something that is a huge difference compared to my conventional LCD laptop and I expect we’ll make more park and beach trips for reading & relaxing now that we have it.

Kindle in the park. [lovingly stolen from Lisa's instagram feed since I forgot to take a picture myself]

Kindle in the park. [lovingly stolen from Lisa’s instagram feed since I forgot to take a picture myself]

I had some reservations about the Kindle, the DRM around their book store and the amount of control that they have over it is of some concern (see FSF for details), but we’ve decided to use it primarily as a side loaded device, where we download and store all our ebooks on our laptops (using a tool like Calibre) and then side load them onto the Kindle, which is easy to do via USB transfer or emailing them to the Kindle itself.

When buying books, I’ll stick to getting them in a downloadable DRM-free format, so that I can copy them to any device in future – this also solves the backup issue for the Kindle, since anything on my laptop or servers is backed up reliably.

Of course I have to try resisting hacking the Kindle and bricking it in some way, or I’ll have an angry fiancée on the warpath, so for now I’ll keep trying to treat it as a book only.  ;-)

Apache Redirect & Varnish

Apache’s mod_rewrite is an awesome module providing all sorts of impressive capabilities, however it’s sometimes overkill when all you want is to redirect an entire site or path to a different one. In these situations, the mod_alias’s Rewrite function is often the ideal solution, with it’s simple syntax.

Redirect permanent /  http://newhost.example.com/
Redirect permanent /test.html http://newhost.example.com/test.php

Unfortunately I’ve found a situation where certain syntax can cause failures in some environments – consider the following:

Redirect permanent /  http://newhost.example.com

This example will work happily when a user is accessing Apache directly, however it will fail horribly when accessing via a Varnish cache, eg:

$ wget oldhost.example.com
 --2012-11-27 11:26:54--  http://oldhost.example.com/
 Resolving oldhost.example.com (oldhost.example.com)... 172.16.1.1
 Connecting to oldhost.example.com (oldhost.example.com)|172.16.1.1|:80... connected.
 HTTP request sent, awaiting response... 301 Moved Permanently
 Location: unspecified
 ERROR: Redirection (301) without location.

This was the source of a lot of headaches since the site still *worked* correctly when connecting directly to it, but kept failing whenever accessed via the varnish caches.

The cause is simply a missing trailing / in the redirect – configuration should actually look like:

Redirect permanent /  http://newhost.example.com/

It’s a nasty trap for the unwary sysadmin, since it works perfectly when accessing the server directly – I’m not entirely sure whether it’s an issue with Apache allowing this syntax, or whether it’s an issue with Varnish not understanding how to proceed – maybe a bit of both.

RFC 2616 isn’t clear, it states that if the abs_path is empty, it’s the same as abs_path being equal to /, and that when requesting a URL with an undefined abs_path, it should be treated as /… but it doesn’t clarify whether an application serving a URL should or shouldn’t append the trailing /.

I guess ideally Apache should be patched to serve up URLs with a trailing / and Varnish should accept redirects with or without it to protect both applications from blame.

Android ICS & the MTP pain

With Android 3.0, Google switched the way of accessing files on Android devices from USB mass storage device behaviour, to Media Transfer Protocol, a standard initially created by Microsoft, but later developed into part of the USB standard for media devices.

For most Android users, new ICS phones are where they will first encounter this new approach to sharing file content. Phones previously upgraded from Android 2.x may still provide the conventional mass storage device behaviour, but any new ICS devices force the option of either MTP or the camera-focused PTP.

MTP is one of those technologies designed with the best of intentions – the fundamental design around it is that it abstracts the underlying filesystem, enabling the phone/device to store the files in whatever way suits it best and then exposing them to the computer.

Unlike the traditional mass storage mount, the files can be available to both the phone and the computer at the same time, whereas previously Android applications would lose access to the “sdcard” storage space when mounted on the computer.

It’s a great idea, except that it’s nowhere near as compatible with computers as a generic mass storage device is. Whilst Windows has native support for MTP, thanks to MS being the original creator of the standard, MacOS has no native support and requires a download, and Linux systems are all kinds of fail when it comes to MTP.

Considering I use a Linux laptop, this situation is very unhelpful – reading online suggests various options (the arch wiki was quite helpful):

  • Use of GNOME with GVFS or KDE’s KIO to connect the storage using the GUI’s device handling functionality. Unfortunately with my tests performed using Nautilus (GNOME’s file manager) on Debian Testing (recent app versions), I wasn’t able to connect to the device, instead experiencing a lot of timeout errors.
  • Linux has a FUSE driver that is able to read MTP, with corresponding usertools called mtpfs – however in my tests, this driver has some serious usability, stability and trust issues, thanks to it repeatedly failing and randomly unmounting, as well as hiding many files that are known to exist.
  • An alternative driver called “go-mtpfs” (written in Go) which provides user space connectivity to MTP Android devices at reasonable speeds and a reasonable UI for mounting the drives.
  • Failing these, a network transfer over WiFi could be an option, such as Samba or SSH. Bluetooth is also a possibility, but the performance is quite poor for numerous file transfers.

I’ve ended up using go-mtpfs which appears to work quite nicely, at least with my ICS Google Nexus phone, however it’s annoying that it’s not in the Debian repositories yet.

Whilst I understand the design decisions and issues solved by Google moving to using MTP rather than USB Mass Storage, the sudden deprecation of a feature that previously existed is a real pain for a user like myself who is wondering why their new phone won’t connect to their computer…

Personally I think Google should have included a legacy USB Mass Storage option for at least another release (ie till Android 5) to have provided time for desktop OS developers to improve the handling of this protocol.

It would also have been useful if they had included a “Install Android Driver” option to expose a small USB mass storage drive (eg 20MB) that includes drivers/apps for common platforms such as MacOS and Linux, so that users could connect and load on drivers without needing

For now, go-mtpfs seems to work OK, meanwhile one of the GVFS/GNOME developers is re-working the MTP handling to properly support Android devices, so hopefully we won’t have the headaches around this protocol for too much longer…

Great server crash of 2012

In a twist of irony, shortly after boarding my flight in Sydney for my trip back to Wellington to escape the heat of the AU summer, my home NZ server crashed due to the massive 30 degree heatwave experienced in Wellington on Christmas day. :-/

I have two NZ servers, my public facing colocation host, and my “home” server which now lives at my parent’s house following my move. The colocation box is nice and comfy in it’s aircon controlled climate, but the home server fluctuates quite significantly thanks to the Wellington climate and it’s geolocation of being in a house rather than a more temperature consistent apartment/office.

After bringing the host back online, Munin showed some pretty scary looking graphs:

localhost flew too close to the sun and plummeted to it's doom

localhost flew too close to the sun and plummeted to his doom

I’ve had problems with the stability of this system in the past. Whilst I mostly resolved this with the upgrades in cooling, there are still the odd occasions of crashing, which appears to be linked with summer months.

The above graphs are interesting since they show a huge climb in disk temperatures, although I still suspect it’s the CPU that lead to the actual system crash occurring – the CPU temperature graphs show a climb up towards 60 degrees, which is the level where I’ve seen many system crashes in the past.

What’s particularly annoying is that all these crashes cause the RAID 6 to trigger a rebuild – I’m unsure as to why exactly this is, I suspect that maybe the CPU hangs in the middle of a disk operation that has written to some disks, but not all.

Having the RAID rebuild after reboot is particularly nasty since it places even more load and effort onto an already overheated system and subjects the array to increased failure risk due to the loss of redundancy. I’d personally consider this a kernel bug, if a disk operation failed, the array should still have a known good state and be able to recover from that – fail only the blocks that are borked.

Other than buying less iffy hardware and finding a cooler spot in the house, there’s not a lot else I can do for this box…. I’m pondering using CPU frequency scaling to help reduce the temperature, by dropping the clock speed of the CPU if it gets too hot, but that has it’s own set of risks and issues associated with it.

In past experiments with temperature scaling on this host, it hasn’t worked too well with the high virtualised workload causing it to swap frequently between high and low performance, leading to an increase in latency and general sluggishness on the host. There’s also a risk that clocking down the CPU may just result in the same work taking longer on the CPU potentially still generating a lot of heat.

I could attack the workload somewhat, the VMs on the host are named based on their role, eg (prod-, devel-, dr-) so there’s the option to make use of KVM to suspend all but key production VMs when the temperature gets too high. Further VM type tagging would help target this a bit more, for example my minecraft VM is a production host, but it’s less important than my file server VM and could be suspended on that basis.

Fundamentally the host  staying online outweighs the importance of any of the workloads, on the simple basis that if the host is still online, it can restart services when needed. If the host is down, then all services are broken until human intervention can be provided.