Monthly Archives: January 2012

So I might have gotten slightly engaged….

You may recall that this time last year I met a very lovely lady and becoming somewhat attached to her. On the 29th of January it was our 1 yr “official” anniversary, which we spent in Wellington.

I realised there’s nobody that I’d rather spend the rest of my life with than her, so last night I asked Lisa to marry me. And she said yes. :’)

Whilst this ring is not as pretty as a Linux server, I think Lisa did approve of this purchase more.

I love you Lisa – spending the rest of my life with you is an amazing, scary, exhilarating feeling all at the same time.

Realising that there’s someone in my life who cares about me as much as I care about them just continues to amaze me every day and I’m so, so happy I met you.

 

So yeah, I now have one very surprised but happy fiancé and about a million tweets from followers, friends and family. As we were in Wellington for the weekend, it worked out well, being able to see and catch up with many of our friends and my side of the family. (Thankfully she said yes, it would have been a very awkward weekend otherwise). ;-)

Still a lot of stuff to figure out and sort out, I had kind of assumed at this stage my work was done,but turns out there are weddings, life plans, cake testing and getting something other than seans to wear to sort out. :-/

We haven’t really made plans yet, but we’re thinking engagement of at least a year, before we make that final step (or she’ll get sick of me and leave by then and I’ll become (more of) an alcoholic reclusive geek), so it would be a wedding in early 2013. Unsure where, but sounds like Wellington or Hawke’s Bay

I know a few of you want to know how it came about, how I proposed, etc, I’m hoping Lisa will blog that in the next day or two along with some horribly cute couplesy pics once we get someone to take a pic that’s more public appropriate than the content of my phone’s camera.

I’m sure I’ll blog something more in the next few days, just wanted to get something up ATM. :-)

Intel 320 SSD stats & encryption

I recently obtained a 120GB Intel 320 SSD for upgrading my Lenovo Thinkpad X201i from it’s sluggish hard disk to something with a bit sharper performance.

Whilst not the latest and highest performance SSD from Intel, it’s certainly still very quick compared to the hard disk, and it made more sense than buying a more expensive newer model that would be restricted by the SATA 2 bus on my laptop.

The performance increase is impressive, my sequential reads went from 40,300 KB/s to 132,673 KB/s, showing dramatically faster boot performance and snappy application load times. And the seek times jumped massively from 151.4/per second to at least 10,524/per second.

Infact, the SSD is so fast, it can be difficult to get stats of it’s true seek performance. With the seeks completing in only a few microseconds, the bonnie++ tests often finished a bit early and the results would vary, it’s possible the seeks might be even larger than 10k+ per second.

The next major question for me, was what would the performance be if running disk encryption ontop of the SSD. Due to the private nature of my data, I fully encrypt my laptop using dm-crypt/Linux disk encryption with AES 256bit, so that if the machine is ever stolen, the data is unreadable.

Of course, this security imposes an overhead – data needs to be decrypted before it can be read, adding additional overheads, particularly with CPU performance. It’s also worth noting, that the Linux disk encryption implementation is single threaded, meaning that the maximum encryption/decryption performance is limited by the maximum performance of a single core of your processor.

After installing the OS using an encrypted disk, there was a noticeable performance drop. In particular, the sequential reads dropped from 132,673 KB/s to a much less exciting 69,805 KB/s. Whilst still significantly faster than the conventional hard drive’s 40,300 KB/s, it’s a big drop from the true capability of the SSD.

Fortunately the write performance was impacted far less, I suspect because the OS and the CPU core doing the encryption was able to keep up with the slower performance of writing to the SSD, in comparison to the reads. Based on the stats I obtained, it looks like my laptop tops out around 70,000 KB/s, so any additional performance of the SSD above that is wasted.

I’ve uploaded the actual performance statistics generated to a separate page, which you can view if interested.

From a usability point-of-view, even with encryption, the boot time performance is impressive, the laptop starts in about half the time of what it did previously, along with massive improvements in the start time of applications.

The improvements are particularly noticeable when loading a number of applications concurrently – with a conventional hard drive, the need to load data across different physical parts of the disk platters causes a lot of delays when multitasking application loads. On the SSD, I can click a number of applications and have them *all* load within a second or two.

Overall I’m pleased with the upgrade, even with the reduced performance from encryption, the SSD still offers some major performance upgrades and was well worth doing.

The only outstanding downside now is the issue of fitting all my data that I actually want to regularly access on my laptop onto the small size of the SSD…. I’m currently looking into filesystems that provide offline access or caching of networked filesystems from my servers, so that I can have regularly accessed files stored locally, but the full selection just a network transfer away.

Great Ocean Roadtrip

The weekend before linux.conf.au, I decided to go on a roadtrip with @chrisjrn down south of Melbourne for a roadtrip through the rural areas and along the Great Ocean Road.

I started the trip with a pickup from @chrisjrn from the Melbourne International Airport, after a short stopover at the Virgin lounge for some food, we headed out to drive through Melbourne CBD and along the coast to Sorrento.

My crazy tasmanian driver, @chrisjrn

Melbourne CBD skyline in the distance

Once at Sorrento, we took the Sorrento-Queensclift ferry from Sorrento to Queenscliff with the car, around a 1 hour trip across the main shipping lane into Melbourne.

Yay, I'm on a boat! (Crossing the Melbourne harbour)

Moar Boats!

After getting off the ferry and negoiating deep fried deliciousness from a local takeaway joint (silly aussies not understanding NZder accents), we hit the road and got onto the Great Ocean Rd.

It’s a pretty neat drive with many twists, turns and other interesting bits. The amount of tourist-specific signs is amusing, with always a sign stating “In Australia, we drive on the left” after every major tourist turnoff.

It also appears that every Australian rural town is required to have a carnival on, I must have passed around 20 of them during this trip.

After driving for some time, we started getting our first views of the Australian coast line, shortly followed with a stop off at the Split Point lighthouse, offering expanded views over the coast.

Start of the Ocean View Rd

View along the coast line from Split Point lighthouse

View out into the far distance from Split Point lighthouse

Split Point Lighthouse

Some dreamy looking tourist. Off-camera: hordes of fine ladies swooning nearby.

Me looking dorky by a lighthouse.

 

Split Point lighthouse plaque

These pillers are littered along the coast and form a major series attraction called the Twelve Apostles further along the road.

As it was getting late in the day (18:00+), we decided to pull in at Lorne and looked for some accommodation. After spending some time looking around, we determined nothing was open or available, before finding a cabin/motel online further along the road, towards Apollo Bay.

Stopped to look for accommodation in Lorne, view from seating area.

We ended up in a cabin up on the hills, with a partial view out over the bay and lots and lots of wildlife around.

Pretty boys came to visit our motel - saw at least 8 at the same time and could hear them running over the roof.

Pretty birdies!

Red birdies!

The next morning we set off to the Twelve Apostles, pillars of sandstone formed by erosion of the clift-face.

I'm Jethro. These are the Twelve Apostles. (or actually 11.5, since one has kind of fallen down now)

I think these huge clifts are to stop the New Zealanders from invading easily.

View platform on exposed clift face that is slowly being worn away... will eventually be an additional apostle.

Awesome signage.

Even moar rocks!

After taking pictures of lots of rocks and my hair moving dreamily in the wind, we headed up to Ballarat for the conference on the rural roads of Victoria.

Rolling through the outback. Well, kinda.

 

Rural Victoria looks very much like a dry Auckland at times, but with the addition of eucalyptus trees.

And that completes my trip around rural Victoria – next up, LCA posts. :-)

linux.conf.au 2012

In a couple days I’ll be flying out to Melbourne, Australia for linux.conf.au 2012, the undisputed greatest week of the year, being held in Ballarat.

I’ve been attending this conference since 2006 in Dunedin and it’s continued to be an amazing eye opener in the world of technology, open source and amazing people – considering when I first attended the conference, straight out of high-school being the only person out of 500+ students interested in technology, to finding that there are hundreds of even more hard core geeks that me, was totally amazing.

I’ll be doing a bit of tripping around like I did last year (see category linux.conf.au) – this time I’ll be spending 2 days before the conference doing a road trip with my mate Chris, followed by another 2 days after the conference where I stay in the Melbourne CBD for  exploring the city in more detail.

Key dates:

  • 14th Jan – Early morning flight from Auckland to Melbourne, Roadtrip with Chris
  • 15th Jan – Roadtrip with Chris, arriving in Ballarat in the afternoon.
  • 16th Jan – Start of linux.conf.au :-D
  • 20th Jan – End of linux.conf.au :'(
  • 21st Jan – Melbourne CBD Adventures
  • 22nd Jan – Melbourne CBD Adventures
  • 23rd Jan – Melbourne CBD Adventures, afternoon flight back to Auckland.

If you’re in Melbourne and want to catch up, let me know via email, twitter or XMPP and I’ll be keen for coffee/beer/seedybar. :-)

apt-get install debian

Early last year I wrote how I was concerned about the progress and future of the CentOS project and was considering other options.

As of today, I’ve now shifted my primary workstation (Lenovo X201i laptop) from the somewhat out of date Fedora 13 over to Debian Stable/Squeeze.

The main drives for this change were:

  • Fedora 13 was out-of-date and lacking security fixes, hence requirement to upgrade.
  • The logical replacement, Fedora 16, now ships with GNOME 3 which I found to be buggy and a regression to my work flow and requirements (not going into details here, but essentially issues with dual screens, workspaces and customization of toolbars).
  • Desire to seriously try Debian as a primary system with the purpose of evaluating it’s suitability as a replacement for my CentOS servers.
  • Requirement for a distribution that made major release upgrades feasible (Fedora can do version jumps, but not recommended, making it a tricky process to find time to do a laptop upgrade/rebuild).
  • Distribution standardisation across my server & workstation environments.
  • I needed a full reinstall in order to downsize from a 320GB HDD to a 120GB SSD.
  • Reliability – my laptop is my primary business machine, if it doesn’t work, I’m going to be living on instant noodles until it starts working. Or even worse, work will buy me a Macbook to use like everyone else. :-/

I chose Debian particularly, since it would be a fine option for replacing my CentOS servers in the future with long life support & stability, it’s large package selection and the fact that it’s committed to freedom and openness (as is Fedora also); all of which made it more attractive than Ubuntu for me, which feels much more desktop and fast release focused.

So far, I’m loving it – the distribution is solid, well built and developer friendly, and the package selection is pretty decent, not to mention apt being nice and snappy (although the SSD sure helps there ;-)

I’ve had a couple minor issues relating to my Lenovo hardware that I’ve been able to resolve and have gotten into building a few Debian packages in order to backport newer versions of programs like Firefox/Iceweasel.

From what I’ve observed with playing with Debian today is that’s a pretty awesome distribution, but not entirely as perfect as some of it’s users like to make out:

  • The installer is a bit dated – not due to the text installer (I fucking love text installers! \m/), but rather due to it’s lack of support for WPA/2 wifi access points and also the ability to allow the user to make broken systems without warning (eg no /boot partition when you don’t have enough coffee like me).
  • Debian is often credited for having all the packages under the sun in it – whilst almost true, I did note that a number of my more obscure package choices didn’t exist, sending me  running for my compiler a few times.
  • It would be nice if stable backports tracked some of the common packages that users like updated on older systems – programs like web browsers, instant messengers and maybe even kernels for uncooperative hardware. A user could avoid this by using Debian testing, but there are valid reasons to use stable + some backports over using testing or unstable.
  • I think rpm has nicer command line options for directly working with installed or to-be-installed packages than dpkg. Having said that, some of this could be user familiarity/likes, so time will tell as I use it more.

Over all though, these are minor issues – I think it’s a fantastic distribution with so much working out of the box, applications appearing well tested and good online documentation and resources.

I’m currently running trials and comparisons of Debian with my CentOS hosts with a view for replacing my current CentOS 5 instances with Debian Stable instances over a phased migration period, as well as testing features like LDAP authentication and KVM, but it’s looking pretty damn good so far.

At this stage I’ve only used CentOS 6 as a KVM host platform and it seems unlikely I’ll end up deploying any CentOS 6 VMs with all the security update release slowness. With only a couple servers on CentOS 6 altogether, I’m pondering switching them over to Debian sooner rather than later to reduce maintenance efforts.

[FYI, this post isn’t intended as an attack/demands at the CentOS developers, rather it’s recognizing they’re a volunteer team and probably lacking resource – and I thank them for their efforts, but it appears long term it’s not a good option for my requirements.]

It does also raise questions about Red Hat’s RHEL future with engineers like myself – with Red Hat no longer offering a free-as-in-beer-with-no-support option and CentOS being too slow, more geeks like myself might move to Debian. And if we do so, when the next enterprise project comes along, will we be recommending RHEL or Debian?

Red Hat offers the advantage of commercial support, but for a company with their own engineers, Debian may be more appropriate and budget friendly.

100% pure freedom phone?

As per my earlier rant about Android’s openness, I’m not particularly happy with all the binary components on my phone, nor am I particularly happy with the Android Market’s control and lack of clarity around licensing.

There’s multiple issues with propietary software and why I’ve always been an advocate for not just open source, but more importantly, software freedom. In particular, I try and structure all my computing environments so that I can:

  • Always customize the applications I use if needed – this could be bug fixes, feature changes, etc.
  • Having advertisement and tracking/spyware free software. I’d rather pay good money for software than have it advertisement supported or selling my information to others.
  • Have no dependence on gatekeepers running centralized services – I prefer to run distributed federated services, such as SMTP, (Email) and XMPP (IM) for communications, rather than relying on proprietary networks (eg imessage, skype).
  • Full control and responsibility for the security and privacy of my own data, rather than outsourcing to cloud providers.

It seems it would be possible to replace most of the proprietary components that Google supplies with open source components, but in a quick search I didn’t find any Android distributions that have this bundled up into an easy packaged solution.

One of the more popular distributions, Cynaogenmod has some nice features and is open source, but isn’t specifically designed to be *only* open source, whereas I want a distribution that focuses on making it easy to find, install and manage open source software only.

So I’m making plans to do a custom build of Android for my phone which will feature only free as in freedom software components, with the exception of the hardware driver binary blobs.

  • Replace Android Market with the all-open source F-Droid application – this market is 100% open source and both the client and server are open source, so you can even start your own market. One particularly good feature, is the ability to install older versions, I’ve been bitten in the past with updates introducing bugs in the past with no rollback.
  • Email is well handled with open standards IMAP and IMAP IDLE – I’ve been using K9 Mail for some time (open source build of Android’s email client with additional tweaks) and it works beautifully. With the IMAP IDLE functionality, my phone gets notified about the new mail message within a few seconds of the mailserver completing the processing of the message through to my inbox.
  • Replace contact sync with an LDAP contact directory and sync tool to go against that. LDAP is supported by most address book applications and is something I want to use for all my contacts to make it easier to move between applications.
  • Obtain an XMPP client to replace google talk with support for any XMPP/Jabber server desired, whether Google or another server. Considering I use my own XMPP server already, this is something that’s been on my list for a while.
  • Use aCal with an open source CalDAV server (such as DAViCal) for sharing calenders between devices.
  • Replace google maps with open street maps.
  • This would also offer the advantage of not needing to use Google’s cloud services for storing my address book information, something I’ve never particularly liked the idea of, but was somewhat forced upon in the early days of Android 1.5.

As part of this change, I would also end up dumping Android Market and going with only open source applications for Android – the downside will of course be less application selection, but the up side would be less crapware, no adware applications and full control to install any version and manage applications better.

And the end result would be a truly free, open source Android OS on my phone, which I have full control over, with all data stored on either my phone or servers under my control.

I’ll be fitting in the work as I get time slowly replacing components till I have a reliable fully open stack on my phone and blogging my progress. :-)

Android Market Immaturity

Whilst I’m on the war path of Android, there’s a number of major issues that the Android Market has which have been causing me great annoyance lately. It feels very much like Google rushed out a Market application that meets their major requirements, but haven’t put much thought into a lot of how the market will behave in the real world.

 

1. Application Update Management

My IT background has a large component of working with enterprise and corporate organisations, in particular, telecommunications companies. These companies are often known for their annoyingly slow habits of deploying new software:

  1. Determine new software version to use.
  2. Document installation, deployment procedures.
  3. Complete strict testing of applications.
  4. Deploy application.
  5. Test and ensure functionality. If a fault occurs, rollback using the documented procedures.

On the other end of the spectrum, the Android Market has the following behavior:

  1. Find updates. (automatic updating can be turned on/off per application).
  2. Install them.
  3. Don’t like the application following the update? Software bugs? Tough, deal with it.

Whilst I’m hardly going to advocate making test plans for deploying Android updates, I think Google need to take some lessons from the enterprise environments – software will always surprise you with bugs, so plan for rollback options.

There will be times when you update an android application, only to discover that it’s changed in some undesirable way, or that it’s developed a bug in a key feature that you use every day or maybe just doesn’t suit you as much as the older release.

I’ve experienced this issue in the past, where a twitter client update broke posting images via twitter for about a month, before a subsequent update fixed it. Whilst this was occurring, I had no means to be able to go and downgrade the application to the older version that had worked fine.

Sure, it’s not as scary an issue as 10,000 customers not having internet like the telco world, but for that user who’s suffering a bug that impacts something they use daily, it’s a big fucking deal.

Add versioning and rollback support. Seriously. Please. Linux has had this sorted for years (decades?), you can always downgrade a package on a distribution to an earlier version if so desired.

Whilst it is possible to downgrade an application on Android if you can locate the .apk file elsewhere, if the application is only available via the Android Market, there is no approach other than earlier phone application backups that you might have created.

 

2. Vanishing Applications

I’ve been using Android for some time now, since around Android 1.5, during this time I’ve used a lot of different applications and have experienced the annoying issue of applications that I like and use being removed from the Android market place.

What tends to happen is:

  1. User find a nice application that meets their needs, downloads and installs.
  2. Developer pulls the application from the market – this can be any number of reasons – trademarking, unhappiness at application quality, removing a free app and going commercial only, no longer any desire to maintain, or even due to removal by Google for malware.
  3. User ends up buying a new phone, or re-installs a new Android OS image and wants to install all their favorite applications again.
  4. User is unable to find their application on the market to download again.

Once this happens, the only option is to try and recover the application from an existing phone, find it floating around online or if it’s an open source application, find the public repository (even abandoned apps tend to keep the source around) and download and compile the application.

Otherwise the user is left with trying to find an alternative application (if one exists) that could be better or worse than what they previously had.

This particular problem has bitten me enough that I’m always actively seeking for open source options and choosing them, even if a proprietary application is slightly better – the knowledge that I can always build the app myself if it vanishes is a key point.

Unfortunately it’s not that easy to always tell which apps are open source or proprietary thanks to the Android Market’s unclear licensing information:

 

3. Clear licensing information.

Android Market will not report what license a particular application has when viewing the applications details or even when downloading the application.

This is a problem as there’s no way in the market application to tell whether an application is free as in freedom or free as in beer, which is a big problem for any users like myself wanting to choose software options that are under an open source license.

There have been numerous requests to Google to change this, something that surely must not be  a hard feature to add, but there’s been no visible progress on this issue.

For now I’m taking more efforts to research applications before installing them, and using F-Droid, the open source only repository as a first stop to find applications.

 

4. Freedom & Censorship

The use of the Google Market application offers some handy features such as the ability to remotely install software onto the phone via browsing the market website, a legitimate and useful function for some.

This connection to Google also allows Google to remove applications that are undesirable – the intent of this is to remove known malware and malicious content from devices, once again, a legitmate and valid use.

The downside, is that there is the capability for Google to use this connection to install or remove other software components in future, for either their own motives or that of a court order.

Consider something like a wikileaks application providing leaked data, or an application to bypass censorship which causes embarrassment or problems for the US governement. As a US company, Google could be ordered to remove that application from devices worldwide, a very plausible and concerning scenario – even if a user is confident about the ethics of Google, it wouldn’t stop a court order forcing software to be removed.

If this scenario seems far fetched, remember that Amazon removed a particular book from all their e-readers after a copyright dispute, removing not only the book, but all the user prepared notes for them.

I’m a strong supporter of computing freedom, having vendors like Google becoming gatekeepers and controllers of what we can and cannot run is concerning, particularly as the future of legislative policy appears to be tighter and nastier, particularly with the US.

 

Can it be fixed?

It would be pretty straightforwards for Google to fix issues 1-3:

  • Add version awareness to the market place, so a user could downgrade applications – even if it was limited so a user could only download to a version they previously had, I would be happy.
  • Keep pulled applications in the market place (with exclusion of apps removed for malware/malicious purposes – in that case, it should be removed and labelled as such) at least for users who have downloaded them in the past, so we can continue to use our favorite apps. A warning that this application has been abandoned or some other term would be fine.
  • Provide licensing information for applications, along with search abilities to find applications by license type. A link to the upstream source would be a nice touch too.

The 4th issue is a little more complex as the ability to remotely manage software has valid features and isn’t as simple as just removing.

Ideally I think the best approach would be to adjust the structure of Google’s Android integration, so the hooks into Google having control over the phone can be changed to allow/always prompt/disable approach.

This still allows for all the current functionality, but gives users with concerns about Google’s abilities to control how their phone behaves.

I’m pessimistic about Google actually going and fixing these things – they aren’t major selling points to attracting new users to Android, but I think they need to be addressed for Android to be more reliable and usable long term.

Android: the free-ish mobile platform

I’ve been using Android for a while now, starting with the somewhat underpowered HTC Magic (G2) before moving up to a much snappier Google/Samsung Nexus S which is now my current model.

Whilst I’m a fan of the platform overall, I’m encountering more and more issues every day with the fact that Android is being positioned as the poster child of open source in the mobile space (with other alternatives like Meego and WebOS way behind in terms of market share and consumer awareness), yet Android is only partially open source, still relying on large proprietary chunks.

With the recent release of Android 4 (Ice Cream Sandwich), I decided I would run through the steps of compiling Android from source code – I’m a firm believer of only running things that you have the ability compile yourself, and have gone through the exercise of building Linux From Scratch and custom distributions in the past to gain understanding of how the Linux OS is assembled and functions.

Android’s open source release (AOSP) is available from source.android.com which provides instructions for downloading the (large!) source tree, tools and building them into a functional device.

Doing so was an interesting experience – some of the first issues encountered are:

  1. AOSP is limited to working out-of-the-box with only a select number of devices – any official “Google” phones, like the Nexus S or Galaxy Nexus are supported, along with a couple additional vendor models (such as the Xoom tablet).
  2. If you have a non-google supported phone, you’re on your own – depending on your vendor, it may be a simplish, painful or maybe impossible task to obtain the required binary blobs. And that doesn’t cover whether or not the phones have locked or unlocked bootloaders.
  3. Even the Google AOSP supported phones can’t run a pure open source stack, proprietary downloads are supplied by Google for specific hardware components for each model and for a specific OS release. Should Google decide to stop supporting a device with future Android versions (as has happened with earlier devices, you won’t easily be able to support the hardware).
  4. The source is big, the build is hungry and you’ll want some performance to build it. I allocated around 40GB for the checked out source and build space and used most of it, along with 8GB of RAM and a few cores on my Phenom. On the plus side, the build only took a few hours, not the days-long efforts some online had predicted.
  5. Google’s build instructions are a bit lacking, given a week, even a Google intern would be able to make a massive improvement to it, I ended up finding many useful commands online that weren’t documented on AOSP’s home page – such as how to package a build into an OTA style .zip for deployment.
  6. The Linux kernel isn’t compiled as part of the Android build process. Instead, Android used a supplied pre-build binary kernel and just includes it into the finished OS image. If you need to adjust the kernel, it must be built separately and then placed into the correct location in the Android sources. This process isn’t documented anywhere on the AOSP homepage that I could find.

The base AOSP build provided me with core functionality including:

  • Functional base operating system and all hardware (thanks to binary blobs from Google for my Nexus S).
  • Communications – calls, txts, wifi, bluetooth, internet browsing
  • Contacts/Address Book.
  • Ability to install applications from direct .apk download or transfer using adb from PC.
  • A generally working and usable device overall.

But I didn’t have a number of needed functions that are typical of off-the-shelf Android devices:

  • No support for Google Account synchronization – without this, I was unable to download synced contacts and any other information from the cloud account.
  • No android market, the only way to install applications is from third party markets, direct download of .apk files or self-compiling applications.
  • No google service-based applications –  google maps, gmail, google talk, etc
  • No face unlock ability – I expected this would be part of ICS, but seems it’s part of Google’s application set…. this mix of having open source and proprietary components is one of the biggest problems with Android, you aren’t always sure what is or isn’t open source.

To get these other needed functions that are typical of an Android phone, you need the Google apps package (or at least the market application so others can be downloaded).

The killer part is that this package isn’t freely available. From what I’ve been able to find, it seems Google only provides their application package to vendors who pass their tests for Android compatibility to maintain quality.

Personally I think this is a lot of crap, considering the shocking quality of some Android devices that have come out – at the very least, the Android Market place application should be freely available, so users can at least choose to download applications, even if Google decides a particular vendor doesn’t deserve their Google apps.

In the end I managed to source a package of the Google applications for ICS thanks to the efforts of the Cyanogenmod team, but this is a shocking approach – not only is there an uncertainty about having the latest versions, but having users trawling through the internet to find tarballs on some forum is an easy avenue for attack and getting malware onto phones.

The fact that I’m so reliant on hackery to get key functionality back for my phone if I choose to build from source, rather than using my phone vendor’s build images is giving me solid reasons to consider the feasibility of dumping Google’s components from my phone and finding open source replacements for them all.

Whilst Google deserves credit for making the base OS comparability easy to build for users of Google-approved devices, the fact that they’ve allowed vendors to get away with binary blobbing drivers everywhere and keeping key functionality proprietary (market, etc) is pretty bad.

Chasing down binary blobs in order to get a device to work as expected is much more reminiscent of days spend pirating software, not of a healthy open source project and it makes Android feel much more hacky and crappy than it should be.

And the fact that the open source build will work with so few of the phones on the market out-of-the-box is just appalling for an OS that’s called open source – I should be able to go pickup any Android phone in the market and be able to compile AOSP for it and have all the hardware supported, not just select models.

Part of this is the fault of device manufacturers, but IMHO, Google should have put down some restrictions in the use of the Android trademark, so that a device could only actually be an “Android” device if it was fully open source and featured an unlocked bootloader.

I’d even accept a compromise where binary blobs are needed for specific hardware, as long as the blob wrapper can be compiled against different kernels and is free to redistribute (aka firmware style), so that I could buy a phone running Android 2 and happily go and build Android 4 for it and expect it to work.

Overall, I’m happy that I could build a functional image for my Nexus S without too much pain, but disappointed that so much of the feature set we are used to with Android isn’t actually open.

Custom CA certificates & Android

With the number of servers I have internally, I have setup my own Certificate Authority and sign all my internal SSL certificates against this private CA.

This offers the useful advantage of being able to import the one CA certificate into all my devices and then being able to validate all connections to remote systems – if you run more than one or two personal servers, I’d highly recommend this approach – certificate signing takes a little bit of getting used to, but it’s a good skill to have.

As I want to access a number of systems via my Android mobile, I needed to import this CA file – the following instructions were followed with ICS release 4.0.3, however it may apply to earlier/later releases as well.

If you’ve followed most typical instructions for building your CA, you will have an PEM encoded CA certificate file in ASCII format. This is fine for import into most browsers and desktop OSes, however Android is particularly fussy with it’s input and requires a binary format only.

You can convert the CA PEM format file with the following command:

openssl x509 -inform PEM -outform DER -in CA.pem -out CA.crt

Then transfer the generated CA.crt file to the sdcard – easiest is via adb:

adb push CA.crt /sdcard/

Once done, you will be able to tell Android to install the CA file via Settings -> Security -> Credential Storage and selecting “Install from storage” and following prompts.

To verify functionality, easiest test is to access an https website signed with your CA certificate via the browser.

Some commenters have had issues - here is me importing a valid CA cert in DER format.

Some commenters have had issues – here is me importing a valid CA cert in DER format.

2012 New Year’s Resolutions

It’s 2012, time to update my new years resolutions and make plans for this year! :-)

Firstly, what’s happened since the start of 2011?

  • I met Lisa, we’ve been together 11+ months now! :-O
  • I’ve packed up my life and moved to Auckland after Lisa started a journalism job up here, it’s the first time I’ve lived outside of Wellington, so it’s a big change for me!
  • I’ve been working for Prophecy for over a year now, these days working from home or from customer sites in Auckland, going pretty well.
  • I’ve done more travelling around NZ than ever before, various road trips around the North Island and seeing new places.

I think I’ve done a pretty good job of the resolutions I made last year:

  • I’ve reduced the amount of stuff I have, sold almost all my rackmount servers, sold all my beloved data centre components and cleared out huge piles of things from the flat.
  • I haven’t quite gotten down to the goals of my less, less, less life approach, now that I’m not single it’s not quite as easy, if I clear out some space, Lisa just fills it in with something.  ;-) Although I’ve done a pretty good job of holding myself off buying gadgets that I don’t really need this year.
  • I’m working sane hours and having a bit more of a work/life balance. Will be interesting to see how this lasts now that I’m working from home though.
  • My blogging is more frequent, although not as much as I think it should be.

What I haven’t achieved is as much projects or geeking as I wanted, so I will be making more time this year for doing so and getting into more fun projects. :-)

So, my new years resolutions for 2012 are:

  • Complete planned minimalism, selling off items I don’t need or use, reducing to things I actually care about. I still have too much cruft IMHO, but it’s hard to keep it low.
  • Getting fitter – I need to lose weight and want to level up my fitness to potentially tackle the Te Araroa walk in late 2012 or early 2013, and with the move to Auckland I’m not getting the level of exercise I used to have in Wellington.
  • Reduce amount of crap in my diet – reducing alcohol and snack foods will be the major difference, if I don’t buy it, I won’t eat it….
  • Get out and explore the Auckland region!
  • Get back up the mountain for some more snowboarding! I loved my first trip and want to do much more!
  • Blog more, social network less. Blogs take longer, but are much more meaningful posts and I waste far too much time on social networks. Target is at least one decent blog post per week.
  • Improve my saving situation and consider investment options for my savings.
  • Move off proprietary networks as much as possible – mostly twitter at this stage, but also interested in options to replace google search, maps, etc.
  • Complete many mostly done projects and release publicly! Lots of cool things I’d like to get out there. :-)
  • Self enhance – work on improving my computing work environment with better tools – I’m planning to try one new tool every week and then pick the best ones for my needs.
  • Learn new computing languages: Python, C++ and Java. I know bits of the above, enough to get by, but I want to be fluent in them.
  • Learn a new IRL language. I want something I can use and then go overseas and live there for a while, so thinking maybe something like French or German, both places I’d like to go. Maybe German, then I can sing along to Rammstein songs better ;-) I think the key bit will be having someone to practice with, ideally a native speaker somewhere.
  • Decide what my long term plans will be – Auckland, Wellington, or overseas. It’s a little more complicated now that I’m not as unattached as I once was.