Author Archives: Jethro Carr

Wynyard Crossing

2 Replies

As part of my mission to get out and explore Auckland some more, I’ve been down at Wynyard Quarter a bit. It’s quite a neat and relatively recent development in Auckland, with several trendy bars/food places and a pedestrian friendly area for walking.

There always some interesting views with the functional industrial port right next to high wealth yachts, along with the trendy outside area and seating. Quite Wellingtonesque in many ways, the Wellington waterfront is always stylish and interesting to wander along, day or night.

Casually chilling on the giant deck chairs

Auckland CBD View

Always interesting ships parked around the place.

One of the cool features of this area is Wynyard Crossing, a double bascule bridge that opens up to allow boat traffic to travel through between the sea and the marina, which opened in August 2011.

I’m kind of a bridge fan, so I couldn’t resist taking a few pictures and a video of the bridge in lift motion.

Idle bridge being idle

Doesn't look toooo steep....

Permission to come on through capt'n!

A goodbye to #geekflat

5 Replies

Whilst I’ve been an official Aucklander for a few months now, I’ve been sort of ignoring the whole end of an era going on with my awesome, amazing time in Wellington at the place known only as #geekflat.

As of mid-feb, the last remaining resident of geekflat, @macropiper will have departed. @thatjohn departed only a few days ago and I left late in late 2011 and am now engaged (like wut?!?!) and living in Auckland.

I’m kind of sad that this stage of my life is over and unlikely to repeat in the same way. I’m pretty nostalgic about it, after I moved to Auckland I missed the guys and times I had there so much that Lisa was asking my friends to talk with me because I was acting so depressed.

I’ve been putting off selling off some of my Wellington-located stuff, like my fridge and other accessories that I had left at #geekflat until now, but with #geekflat being dissolved in February, I need to clear out a few large items that I can’t store easily. (so totally go bid please! ;-)

The actual story of #geekflat arguably started in 2008 when @macropiper and I started looking for our first flat together. I didn’t really know Tom back then, we were introduced by Tom’s employer at the time who I tended to have semi-regular coffee catchups with.

Tom and I were both living at home at the time and I wanted to find a flat with equally geeky people. Tom and I spent a bit of time wandering the streets of Wellington, discovering how crappy the rental market is, before finding a place located in Kingston, a few blocks from my parent’s place. Typical of NZ cheap(ish) rentals, it was lacking insulation, not exactly flashy, but it had it’s charm and was a great starter flat for us.

At the time of leaving home, my collection of stuff was actually quite survivable and really did fit into a car.

Tom and our computers on the day that we first moved into our first flat. Note the cardboard tower that supported the 3G modem so we could get data - any lower would kill data reception.

After getting a little more settled we ended up with a few modern comforts. Like DSL, Playstation3, server rack....

My dreamy self back in 2008. The lovely windows behind me used to leak heat like a sieve in winter and had an awesome ability to let whistling drafts in.

Amazingly we never managed to kill each other – in hindsight, having both of us in the lounge with our computers all the time was bound to lead to some clashes, since we never really could escape each other, but we got over a couple of our arguments OK and I’m always thankful that we became such good mates. Particularly since I’m not the easiest person to live with. ;-)

In late 2009 we decided to move to somewhere that was warmer and also better located to the Wellington CBD – by that stage I was working from home on Amberdms and didn’t drive, so wanted somewhere I could walk in to the CBD for any meetings. Tom and I also agreed we wanted a third person to help balance out the dynamics a bit more and we ended up finding another somewhat geeky guy to move in with us.

And so #geekflat was born, with an amazing location on Thompson St, Mt Cook, it had the best combination of city living with suburban quietness, with only a short walk to get to Manners Mall or even Lambton Quay.

To make things even better, it was at the top of a hill, ensuring some pretty awesome views, but also enforcing exercise for the somewhat unfit geek residents, giving me a pretty good workout on a daily basis.

View out from my bedroom at #geekflat

Sophisticated inter-room connectivity for flatmates to my server. (we did later tape this up over all the door frames to make it a bit neater....)

Whilst at our first flat, I had amassed a few servers and other bits of equipment – moving to #geekflat only made this worse. After a while with the addition of the Amberdms office, I ended up moving my lab to a separate office room using 2x 42U server racks. Whilst it was expensive and time consuming, I really do miss the fun it provided.

A reasonably tidy snapshot of how I spent my time at #geekflat....

The move to #geekflat is also the time of a lot of change in my social life, I went from being a quiet geek not doing much other than computers in the evening to being quite social and getting out – many thanks in part to twitter.

Really this time was my shift from teenager into adulthood, getting more confidence in social situations, meeting lots of people, taking home random people after a night out, learning to drive, realizing I can do pretty much anything I want and making my own choices about how I wanted to live.

Friends hanging at #geekflat - note John on left, Tom in immediate forground, and my server stack on the floor with a same of my staple liquid diet at the time.

Twitter friends at #geekflat. The items on the fridge are hard disk platters.

Whilst #geekflat had a clearly defined computer geek element to it, there was also just a bit of good geeky fun from time to time, particularly thanks to Tom’s quirky nature and my ability to get excited like a 5 yr old.

Tom and lazers was a common #geekflat theme.

What do you mean your flat doesn't have a periodic table of elements shower curtain!??!?!

One should always have lolcat fridge magnets!

Silly amusing stickers and magnets? CHECK! :-D

Not to mention the completely random-yet-awesome things that occurred from time to time, the wellington snow, the weird packages, the cute little mice that ran all over the ethernet cables and ate all my chocolate…

Winter 2011- first time I've ever seen snow in Wellington on regular streets.

The time that someone looked up my whois information and sent novelty boobs to my parent's place addressed to me :-/

During my time at #geekflat I was working on Amberdms, my startup open source & IT services company. Looking back, I’m still undecided whether it was the greatest or worst of times – I think a bit of both.

I loved what I was doing, the products, the people and most of all, the ideas and drive to produce something amazing. But at the same time, the heavy workload and 60-80 hr weeks were taking a toll on me and really put a lot of pressure on me leading to a big depressive downward spiral.

Not to mention that startups are financially hard – I was self-financing the company which certainly added pressure and challenges, particularly for someone previously accustomed to good IT job money and needing to learn to keep expenses and living costs minimal.

Amberdms could almost do with it’s own blog post, but it ties in with #geekflat so much I kind of need to mention it here a bit – after all, it was my fulltime job for several years and Tom was working with me for almost a year as well at one stage.

Looking all professional at Amberdms's shiny new office.

Working hard from the couch in some stylish brandware

My good friend Tom was crazy enough to come work for me AND to continue to flat with me. Good times. :-)

Katipo was our office-away-from-the-office, serving up delicious cheesy fries, nachoes, vege burgers and amazing iced mochas. Sadly it closed down around a year ago now :'(

The Amberdms Team - Tom, Jethro, Bex, out at a tweetup event. (please forgive the terrible photo, the iphone that someone used does not like dark rooms with red styling)

During all this time I was actively seeing a few people – being lonely and suffering rejection certainly doesn’t help with any depressive tendencies. It wasn’t all bad though, I meet a few fun people and had good times and bad. Just a bit too much unrequited feelings that still kind of hurt at times.

Plus it means that Bex has plenty of stories for anyone wanting a laugh at my expense about stupid questions I asked her about things girls do, being the closet female I could find to ask for explanations. ;-)

Amusing gift sent to me by a twitter friend, quite appropriate for this stage of my life.

In late 2010 I decided to take a break from Amberdms and returned to work for my previous employer. Suddenly I had lots more free time and went through a bit of a transformation, getting out more, focusing on just enjoying myself.

I had a bit of fun – got myself some new computing toys, went out a bit more, had a few flings, watched a lot of sci-fi and geeked around with friends.

Tom got a new job with a web development company, in many ways Amberdms worked well for him to get a different perspective and idea of what jobs to look for. Thankfully the stresses of startup life never damaged our friendship. :-)

New haircut, new start right?

Fuck yeah vegetarianism!

Delicious geekflat cookies!

With increased budget, came more delicious cola ;-)

Fuck yeah, epic geeking!

Fuck yeah delicious curry! (I must have been responsible for about 10% of Little India's annual income)

Whilst all this fun was going on, suddenly the unimaginable happened – I met Lisa and things changed a bit from there on….

Oh hai there! Would you like a pikelet?

Uh guys? I think there's a girl in the flat...

And with that, #geekflat started on the path to it’s demise – sadly it just wasn’t big enough for Lisa to live there with me, so once our relationship started, we were doomed to having to move.

Lisa then managed to obtain a journalism/writing job up in Auckland, to which I crazily agreed to move to Auckland with her and then managed to accidentally get engaged.

Being the traveler he is, John left the flat in early 2011, returning again later in the year and moved back in taking over my place in the flat, but it was never intended to be a long term thing – and without all three of us together, there just wasn’t that same dynamic.

So now I’m living in Auckland, John has left #geekflat for another overseas adventure and Tom is moving in with a good friend of his. End of an era. But I don’t regret any of it, just wish that I could relive parts of it at times.

#geekflat may be dead now, but who knows what the future will offer all of us, I hope that the void is filled by other great new experiences and friendships.

The peak #geekflat crowd - Lisa, John, Mitchell (my brother) and Tom

No more server racks, now down to a much more appropiate tower machine providing KVM VMs.

So I might have gotten slightly engaged….

7 Replies

You may recall that this time last year I met a very lovely lady and becoming somewhat attached to her. On the 29th of January it was our 1 yr “official” anniversary, which we spent in Wellington.

I realised there’s nobody that I’d rather spend the rest of my life with than her, so last night I asked Lisa to marry me. And she said yes. :’)

Whilst this ring is not as pretty as a Linux server, I think Lisa did approve of this purchase more.

I love you Lisa – spending the rest of my life with you is an amazing, scary, exhilarating feeling all at the same time.

Realising that there’s someone in my life who cares about me as much as I care about them just continues to amaze me every day and I’m so, so happy I met you.

 

So yeah, I now have one very surprised but happy fiancé and about a million tweets from followers, friends and family. As we were in Wellington for the weekend, it worked out well, being able to see and catch up with many of our friends and my side of the family. (Thankfully she said yes, it would have been a very awkward weekend otherwise). ;-)

Still a lot of stuff to figure out and sort out, I had kind of assumed at this stage my work was done,but turns out there are weddings, life plans, cake testing and getting something other than seans to wear to sort out. :-/

We haven’t really made plans yet, but we’re thinking engagement of at least a year, before we make that final step (or she’ll get sick of me and leave by then and I’ll become (more of) an alcoholic reclusive geek), so it would be a wedding in early 2013. Unsure where, but sounds like Wellington or Hawke’s Bay

I know a few of you want to know how it came about, how I proposed, etc, I’m hoping Lisa will blog that in the next day or two along with some horribly cute couplesy pics once we get someone to take a pic that’s more public appropriate than the content of my phone’s camera.

I’m sure I’ll blog something more in the next few days, just wanted to get something up ATM. :-)

Intel 320 SSD stats & encryption

8 Replies

I recently obtained a 120GB Intel 320 SSD for upgrading my Lenovo Thinkpad X201i from it’s sluggish hard disk to something with a bit sharper performance.

Whilst not the latest and highest performance SSD from Intel, it’s certainly still very quick compared to the hard disk, and it made more sense than buying a more expensive newer model that would be restricted by the SATA 2 bus on my laptop.

The performance increase is impressive, my sequential reads went from 40,300 KB/s to 132,673 KB/s, showing dramatically faster boot performance and snappy application load times. And the seek times jumped massively from 151.4/per second to at least 10,524/per second.

Infact, the SSD is so fast, it can be difficult to get stats of it’s true seek performance. With the seeks completing in only a few microseconds, the bonnie++ tests often finished a bit early and the results would vary, it’s possible the seeks might be even larger than 10k+ per second.

The next major question for me, was what would the performance be if running disk encryption ontop of the SSD. Due to the private nature of my data, I fully encrypt my laptop using dm-crypt/Linux disk encryption with AES 256bit, so that if the machine is ever stolen, the data is unreadable.

Of course, this security imposes an overhead – data needs to be decrypted before it can be read, adding additional overheads, particularly with CPU performance. It’s also worth noting, that the Linux disk encryption implementation is single threaded, meaning that the maximum encryption/decryption performance is limited by the maximum performance of a single core of your processor.

After installing the OS using an encrypted disk, there was a noticeable performance drop. In particular, the sequential reads dropped from 132,673 KB/s to a much less exciting 69,805 KB/s. Whilst still significantly faster than the conventional hard drive’s 40,300 KB/s, it’s a big drop from the true capability of the SSD.

Fortunately the write performance was impacted far less, I suspect because the OS and the CPU core doing the encryption was able to keep up with the slower performance of writing to the SSD, in comparison to the reads. Based on the stats I obtained, it looks like my laptop tops out around 70,000 KB/s, so any additional performance of the SSD above that is wasted.

I’ve uploaded the actual performance statistics generated to a separate page, which you can view if interested.

From a usability point-of-view, even with encryption, the boot time performance is impressive, the laptop starts in about half the time of what it did previously, along with massive improvements in the start time of applications.

The improvements are particularly noticeable when loading a number of applications concurrently – with a conventional hard drive, the need to load data across different physical parts of the disk platters causes a lot of delays when multitasking application loads. On the SSD, I can click a number of applications and have them *all* load within a second or two.

Overall I’m pleased with the upgrade, even with the reduced performance from encryption, the SSD still offers some major performance upgrades and was well worth doing.

The only outstanding downside now is the issue of fitting all my data that I actually want to regularly access on my laptop onto the small size of the SSD…. I’m currently looking into filesystems that provide offline access or caching of networked filesystems from my servers, so that I can have regularly accessed files stored locally, but the full selection just a network transfer away.

Great Ocean Roadtrip

Leave a reply

The weekend before linux.conf.au, I decided to go on a roadtrip with @chrisjrn down south of Melbourne for a roadtrip through the rural areas and along the Great Ocean Road.

I started the trip with a pickup from @chrisjrn from the Melbourne International Airport, after a short stopover at the Virgin lounge for some food, we headed out to drive through Melbourne CBD and along the coast to Sorrento.

My crazy tasmanian driver, @chrisjrn

Melbourne CBD skyline in the distance

Once at Sorrento, we took the Sorrento-Queensclift ferry from Sorrento to Queenscliff with the car, around a 1 hour trip across the main shipping lane into Melbourne.

Yay, I'm on a boat! (Crossing the Melbourne harbour)

Moar Boats!

After getting off the ferry and negoiating deep fried deliciousness from a local takeaway joint (silly aussies not understanding NZder accents), we hit the road and got onto the Great Ocean Rd.

It’s a pretty neat drive with many twists, turns and other interesting bits. The amount of tourist-specific signs is amusing, with always a sign stating “In Australia, we drive on the left” after every major tourist turnoff.

It also appears that every Australian rural town is required to have a carnival on, I must have passed around 20 of them during this trip.

After driving for some time, we started getting our first views of the Australian coast line, shortly followed with a stop off at the Split Point lighthouse, offering expanded views over the coast.

Start of the Ocean View Rd

View along the coast line from Split Point lighthouse

View out into the far distance from Split Point lighthouse

Split Point Lighthouse

Some dreamy looking tourist. Off-camera: hordes of fine ladies swooning nearby.

Me looking dorky by a lighthouse.

 

Split Point lighthouse plaque

These pillers are littered along the coast and form a major series attraction called the Twelve Apostles further along the road.

As it was getting late in the day (18:00+), we decided to pull in at Lorne and looked for some accommodation. After spending some time looking around, we determined nothing was open or available, before finding a cabin/motel online further along the road, towards Apollo Bay.

Stopped to look for accommodation in Lorne, view from seating area.

We ended up in a cabin up on the hills, with a partial view out over the bay and lots and lots of wildlife around.

Pretty boys came to visit our motel - saw at least 8 at the same time and could hear them running over the roof.

Pretty birdies!

Red birdies!

The next morning we set off to the Twelve Apostles, pillars of sandstone formed by erosion of the clift-face.

I'm Jethro. These are the Twelve Apostles. (or actually 11.5, since one has kind of fallen down now)

I think these huge clifts are to stop the New Zealanders from invading easily.

View platform on exposed clift face that is slowly being worn away... will eventually be an additional apostle.

Awesome signage.

Even moar rocks!

After taking pictures of lots of rocks and my hair moving dreamily in the wind, we headed up to Ballarat for the conference on the rural roads of Victoria.

Rolling through the outback. Well, kinda.

 

Rural Victoria looks very much like a dry Auckland at times, but with the addition of eucalyptus trees.

And that completes my trip around rural Victoria – next up, LCA posts. :-)

linux.conf.au 2012

Leave a reply

In a couple days I’ll be flying out to Melbourne, Australia for linux.conf.au 2012, the undisputed greatest week of the year, being held in Ballarat.

I’ve been attending this conference since 2006 in Dunedin and it’s continued to be an amazing eye opener in the world of technology, open source and amazing people – considering when I first attended the conference, straight out of high-school being the only person out of 500+ students interested in technology, to finding that there are hundreds of even more hard core geeks that me, was totally amazing.

I’ll be doing a bit of tripping around like I did last year (see category linux.conf.au) – this time I’ll be spending 2 days before the conference doing a road trip with my mate Chris, followed by another 2 days after the conference where I stay in the Melbourne CBD for  exploring the city in more detail.

Key dates:

  • 14th Jan – Early morning flight from Auckland to Melbourne, Roadtrip with Chris
  • 15th Jan – Roadtrip with Chris, arriving in Ballarat in the afternoon.
  • 16th Jan – Start of linux.conf.au :-D
  • 20th Jan – End of linux.conf.au :'(
  • 21st Jan – Melbourne CBD Adventures
  • 22nd Jan – Melbourne CBD Adventures
  • 23rd Jan – Melbourne CBD Adventures, afternoon flight back to Auckland.

If you’re in Melbourne and want to catch up, let me know via email, twitter or XMPP and I’ll be keen for coffee/beer/seedybar. :-)

apt-get install debian

11 Replies

Early last year I wrote how I was concerned about the progress and future of the CentOS project and was considering other options.

As of today, I’ve now shifted my primary workstation (Lenovo X201i laptop) from the somewhat out of date Fedora 13 over to Debian Stable/Squeeze.

The main drives for this change were:

  • Fedora 13 was out-of-date and lacking security fixes, hence requirement to upgrade.
  • The logical replacement, Fedora 16, now ships with GNOME 3 which I found to be buggy and a regression to my work flow and requirements (not going into details here, but essentially issues with dual screens, workspaces and customization of toolbars).
  • Desire to seriously try Debian as a primary system with the purpose of evaluating it’s suitability as a replacement for my CentOS servers.
  • Requirement for a distribution that made major release upgrades feasible (Fedora can do version jumps, but not recommended, making it a tricky process to find time to do a laptop upgrade/rebuild).
  • Distribution standardisation across my server & workstation environments.
  • I needed a full reinstall in order to downsize from a 320GB HDD to a 120GB SSD.
  • Reliability – my laptop is my primary business machine, if it doesn’t work, I’m going to be living on instant noodles until it starts working. Or even worse, work will buy me a Macbook to use like everyone else. :-/

I chose Debian particularly, since it would be a fine option for replacing my CentOS servers in the future with long life support & stability, it’s large package selection and the fact that it’s committed to freedom and openness (as is Fedora also); all of which made it more attractive than Ubuntu for me, which feels much more desktop and fast release focused.

So far, I’m loving it – the distribution is solid, well built and developer friendly, and the package selection is pretty decent, not to mention apt being nice and snappy (although the SSD sure helps there ;-)

I’ve had a couple minor issues relating to my Lenovo hardware that I’ve been able to resolve and have gotten into building a few Debian packages in order to backport newer versions of programs like Firefox/Iceweasel.

From what I’ve observed with playing with Debian today is that’s a pretty awesome distribution, but not entirely as perfect as some of it’s users like to make out:

  • The installer is a bit dated – not due to the text installer (I fucking love text installers! \m/), but rather due to it’s lack of support for WPA/2 wifi access points and also the ability to allow the user to make broken systems without warning (eg no /boot partition when you don’t have enough coffee like me).
  • Debian is often credited for having all the packages under the sun in it – whilst almost true, I did note that a number of my more obscure package choices didn’t exist, sending me  running for my compiler a few times.
  • It would be nice if stable backports tracked some of the common packages that users like updated on older systems – programs like web browsers, instant messengers and maybe even kernels for uncooperative hardware. A user could avoid this by using Debian testing, but there are valid reasons to use stable + some backports over using testing or unstable.
  • I think rpm has nicer command line options for directly working with installed or to-be-installed packages than dpkg. Having said that, some of this could be user familiarity/likes, so time will tell as I use it more.

Over all though, these are minor issues – I think it’s a fantastic distribution with so much working out of the box, applications appearing well tested and good online documentation and resources.

I’m currently running trials and comparisons of Debian with my CentOS hosts with a view for replacing my current CentOS 5 instances with Debian Stable instances over a phased migration period, as well as testing features like LDAP authentication and KVM, but it’s looking pretty damn good so far.

At this stage I’ve only used CentOS 6 as a KVM host platform and it seems unlikely I’ll end up deploying any CentOS 6 VMs with all the security update release slowness. With only a couple servers on CentOS 6 altogether, I’m pondering switching them over to Debian sooner rather than later to reduce maintenance efforts.

[FYI, this post isn’t intended as an attack/demands at the CentOS developers, rather it’s recognizing they’re a volunteer team and probably lacking resource – and I thank them for their efforts, but it appears long term it’s not a good option for my requirements.]

It does also raise questions about Red Hat’s RHEL future with engineers like myself – with Red Hat no longer offering a free-as-in-beer-with-no-support option and CentOS being too slow, more geeks like myself might move to Debian. And if we do so, when the next enterprise project comes along, will we be recommending RHEL or Debian?

Red Hat offers the advantage of commercial support, but for a company with their own engineers, Debian may be more appropriate and budget friendly.

100% pure freedom phone?

2 Replies

As per my earlier rant about Android’s openness, I’m not particularly happy with all the binary components on my phone, nor am I particularly happy with the Android Market’s control and lack of clarity around licensing.

There’s multiple issues with propietary software and why I’ve always been an advocate for not just open source, but more importantly, software freedom. In particular, I try and structure all my computing environments so that I can:

  • Always customize the applications I use if needed – this could be bug fixes, feature changes, etc.
  • Having advertisement and tracking/spyware free software. I’d rather pay good money for software than have it advertisement supported or selling my information to others.
  • Have no dependence on gatekeepers running centralized services – I prefer to run distributed federated services, such as SMTP, (Email) and XMPP (IM) for communications, rather than relying on proprietary networks (eg imessage, skype).
  • Full control and responsibility for the security and privacy of my own data, rather than outsourcing to cloud providers.

It seems it would be possible to replace most of the proprietary components that Google supplies with open source components, but in a quick search I didn’t find any Android distributions that have this bundled up into an easy packaged solution.

One of the more popular distributions, Cynaogenmod has some nice features and is open source, but isn’t specifically designed to be *only* open source, whereas I want a distribution that focuses on making it easy to find, install and manage open source software only.

So I’m making plans to do a custom build of Android for my phone which will feature only free as in freedom software components, with the exception of the hardware driver binary blobs.

  • Replace Android Market with the all-open source F-Droid application – this market is 100% open source and both the client and server are open source, so you can even start your own market. One particularly good feature, is the ability to install older versions, I’ve been bitten in the past with updates introducing bugs in the past with no rollback.
  • Email is well handled with open standards IMAP and IMAP IDLE – I’ve been using K9 Mail for some time (open source build of Android’s email client with additional tweaks) and it works beautifully. With the IMAP IDLE functionality, my phone gets notified about the new mail message within a few seconds of the mailserver completing the processing of the message through to my inbox.
  • Replace contact sync with an LDAP contact directory and sync tool to go against that. LDAP is supported by most address book applications and is something I want to use for all my contacts to make it easier to move between applications.
  • Obtain an XMPP client to replace google talk with support for any XMPP/Jabber server desired, whether Google or another server. Considering I use my own XMPP server already, this is something that’s been on my list for a while.
  • Use aCal with an open source CalDAV server (such as DAViCal) for sharing calenders between devices.
  • Replace google maps with open street maps.
  • This would also offer the advantage of not needing to use Google’s cloud services for storing my address book information, something I’ve never particularly liked the idea of, but was somewhat forced upon in the early days of Android 1.5.

As part of this change, I would also end up dumping Android Market and going with only open source applications for Android – the downside will of course be less application selection, but the up side would be less crapware, no adware applications and full control to install any version and manage applications better.

And the end result would be a truly free, open source Android OS on my phone, which I have full control over, with all data stored on either my phone or servers under my control.

I’ll be fitting in the work as I get time slowly replacing components till I have a reliable fully open stack on my phone and blogging my progress. :-)

Android Market Immaturity

Leave a reply

Whilst I’m on the war path of Android, there’s a number of major issues that the Android Market has which have been causing me great annoyance lately. It feels very much like Google rushed out a Market application that meets their major requirements, but haven’t put much thought into a lot of how the market will behave in the real world.

 

1. Application Update Management

My IT background has a large component of working with enterprise and corporate organisations, in particular, telecommunications companies. These companies are often known for their annoyingly slow habits of deploying new software:

  1. Determine new software version to use.
  2. Document installation, deployment procedures.
  3. Complete strict testing of applications.
  4. Deploy application.
  5. Test and ensure functionality. If a fault occurs, rollback using the documented procedures.

On the other end of the spectrum, the Android Market has the following behavior:

  1. Find updates. (automatic updating can be turned on/off per application).
  2. Install them.
  3. Don’t like the application following the update? Software bugs? Tough, deal with it.

Whilst I’m hardly going to advocate making test plans for deploying Android updates, I think Google need to take some lessons from the enterprise environments – software will always surprise you with bugs, so plan for rollback options.

There will be times when you update an android application, only to discover that it’s changed in some undesirable way, or that it’s developed a bug in a key feature that you use every day or maybe just doesn’t suit you as much as the older release.

I’ve experienced this issue in the past, where a twitter client update broke posting images via twitter for about a month, before a subsequent update fixed it. Whilst this was occurring, I had no means to be able to go and downgrade the application to the older version that had worked fine.

Sure, it’s not as scary an issue as 10,000 customers not having internet like the telco world, but for that user who’s suffering a bug that impacts something they use daily, it’s a big fucking deal.

Add versioning and rollback support. Seriously. Please. Linux has had this sorted for years (decades?), you can always downgrade a package on a distribution to an earlier version if so desired.

Whilst it is possible to downgrade an application on Android if you can locate the .apk file elsewhere, if the application is only available via the Android Market, there is no approach other than earlier phone application backups that you might have created.

 

2. Vanishing Applications

I’ve been using Android for some time now, since around Android 1.5, during this time I’ve used a lot of different applications and have experienced the annoying issue of applications that I like and use being removed from the Android market place.

What tends to happen is:

  1. User find a nice application that meets their needs, downloads and installs.
  2. Developer pulls the application from the market – this can be any number of reasons – trademarking, unhappiness at application quality, removing a free app and going commercial only, no longer any desire to maintain, or even due to removal by Google for malware.
  3. User ends up buying a new phone, or re-installs a new Android OS image and wants to install all their favorite applications again.
  4. User is unable to find their application on the market to download again.

Once this happens, the only option is to try and recover the application from an existing phone, find it floating around online or if it’s an open source application, find the public repository (even abandoned apps tend to keep the source around) and download and compile the application.

Otherwise the user is left with trying to find an alternative application (if one exists) that could be better or worse than what they previously had.

This particular problem has bitten me enough that I’m always actively seeking for open source options and choosing them, even if a proprietary application is slightly better – the knowledge that I can always build the app myself if it vanishes is a key point.

Unfortunately it’s not that easy to always tell which apps are open source or proprietary thanks to the Android Market’s unclear licensing information:

 

3. Clear licensing information.

Android Market will not report what license a particular application has when viewing the applications details or even when downloading the application.

This is a problem as there’s no way in the market application to tell whether an application is free as in freedom or free as in beer, which is a big problem for any users like myself wanting to choose software options that are under an open source license.

There have been numerous requests to Google to change this, something that surely must not be  a hard feature to add, but there’s been no visible progress on this issue.

For now I’m taking more efforts to research applications before installing them, and using F-Droid, the open source only repository as a first stop to find applications.

 

4. Freedom & Censorship

The use of the Google Market application offers some handy features such as the ability to remotely install software onto the phone via browsing the market website, a legitimate and useful function for some.

This connection to Google also allows Google to remove applications that are undesirable – the intent of this is to remove known malware and malicious content from devices, once again, a legitmate and valid use.

The downside, is that there is the capability for Google to use this connection to install or remove other software components in future, for either their own motives or that of a court order.

Consider something like a wikileaks application providing leaked data, or an application to bypass censorship which causes embarrassment or problems for the US governement. As a US company, Google could be ordered to remove that application from devices worldwide, a very plausible and concerning scenario – even if a user is confident about the ethics of Google, it wouldn’t stop a court order forcing software to be removed.

If this scenario seems far fetched, remember that Amazon removed a particular book from all their e-readers after a copyright dispute, removing not only the book, but all the user prepared notes for them.

I’m a strong supporter of computing freedom, having vendors like Google becoming gatekeepers and controllers of what we can and cannot run is concerning, particularly as the future of legislative policy appears to be tighter and nastier, particularly with the US.

 

Can it be fixed?

It would be pretty straightforwards for Google to fix issues 1-3:

  • Add version awareness to the market place, so a user could downgrade applications – even if it was limited so a user could only download to a version they previously had, I would be happy.
  • Keep pulled applications in the market place (with exclusion of apps removed for malware/malicious purposes – in that case, it should be removed and labelled as such) at least for users who have downloaded them in the past, so we can continue to use our favorite apps. A warning that this application has been abandoned or some other term would be fine.
  • Provide licensing information for applications, along with search abilities to find applications by license type. A link to the upstream source would be a nice touch too.

The 4th issue is a little more complex as the ability to remotely manage software has valid features and isn’t as simple as just removing.

Ideally I think the best approach would be to adjust the structure of Google’s Android integration, so the hooks into Google having control over the phone can be changed to allow/always prompt/disable approach.

This still allows for all the current functionality, but gives users with concerns about Google’s abilities to control how their phone behaves.

I’m pessimistic about Google actually going and fixing these things – they aren’t major selling points to attracting new users to Android, but I think they need to be addressed for Android to be more reliable and usable long term.

Android: the free-ish mobile platform

5 Replies

I’ve been using Android for a while now, starting with the somewhat underpowered HTC Magic (G2) before moving up to a much snappier Google/Samsung Nexus S which is now my current model.

Whilst I’m a fan of the platform overall, I’m encountering more and more issues every day with the fact that Android is being positioned as the poster child of open source in the mobile space (with other alternatives like Meego and WebOS way behind in terms of market share and consumer awareness), yet Android is only partially open source, still relying on large proprietary chunks.

With the recent release of Android 4 (Ice Cream Sandwich), I decided I would run through the steps of compiling Android from source code – I’m a firm believer of only running things that you have the ability compile yourself, and have gone through the exercise of building Linux From Scratch and custom distributions in the past to gain understanding of how the Linux OS is assembled and functions.

Android’s open source release (AOSP) is available from source.android.com which provides instructions for downloading the (large!) source tree, tools and building them into a functional device.

Doing so was an interesting experience – some of the first issues encountered are:

  1. AOSP is limited to working out-of-the-box with only a select number of devices – any official “Google” phones, like the Nexus S or Galaxy Nexus are supported, along with a couple additional vendor models (such as the Xoom tablet).
  2. If you have a non-google supported phone, you’re on your own – depending on your vendor, it may be a simplish, painful or maybe impossible task to obtain the required binary blobs. And that doesn’t cover whether or not the phones have locked or unlocked bootloaders.
  3. Even the Google AOSP supported phones can’t run a pure open source stack, proprietary downloads are supplied by Google for specific hardware components for each model and for a specific OS release. Should Google decide to stop supporting a device with future Android versions (as has happened with earlier devices, you won’t easily be able to support the hardware).
  4. The source is big, the build is hungry and you’ll want some performance to build it. I allocated around 40GB for the checked out source and build space and used most of it, along with 8GB of RAM and a few cores on my Phenom. On the plus side, the build only took a few hours, not the days-long efforts some online had predicted.
  5. Google’s build instructions are a bit lacking, given a week, even a Google intern would be able to make a massive improvement to it, I ended up finding many useful commands online that weren’t documented on AOSP’s home page – such as how to package a build into an OTA style .zip for deployment.
  6. The Linux kernel isn’t compiled as part of the Android build process. Instead, Android used a supplied pre-build binary kernel and just includes it into the finished OS image. If you need to adjust the kernel, it must be built separately and then placed into the correct location in the Android sources. This process isn’t documented anywhere on the AOSP homepage that I could find.

The base AOSP build provided me with core functionality including:

  • Functional base operating system and all hardware (thanks to binary blobs from Google for my Nexus S).
  • Communications – calls, txts, wifi, bluetooth, internet browsing
  • Contacts/Address Book.
  • Ability to install applications from direct .apk download or transfer using adb from PC.
  • A generally working and usable device overall.

But I didn’t have a number of needed functions that are typical of off-the-shelf Android devices:

  • No support for Google Account synchronization – without this, I was unable to download synced contacts and any other information from the cloud account.
  • No android market, the only way to install applications is from third party markets, direct download of .apk files or self-compiling applications.
  • No google service-based applications –  google maps, gmail, google talk, etc
  • No face unlock ability – I expected this would be part of ICS, but seems it’s part of Google’s application set…. this mix of having open source and proprietary components is one of the biggest problems with Android, you aren’t always sure what is or isn’t open source.

To get these other needed functions that are typical of an Android phone, you need the Google apps package (or at least the market application so others can be downloaded).

The killer part is that this package isn’t freely available. From what I’ve been able to find, it seems Google only provides their application package to vendors who pass their tests for Android compatibility to maintain quality.

Personally I think this is a lot of crap, considering the shocking quality of some Android devices that have come out – at the very least, the Android Market place application should be freely available, so users can at least choose to download applications, even if Google decides a particular vendor doesn’t deserve their Google apps.

In the end I managed to source a package of the Google applications for ICS thanks to the efforts of the Cyanogenmod team, but this is a shocking approach – not only is there an uncertainty about having the latest versions, but having users trawling through the internet to find tarballs on some forum is an easy avenue for attack and getting malware onto phones.

The fact that I’m so reliant on hackery to get key functionality back for my phone if I choose to build from source, rather than using my phone vendor’s build images is giving me solid reasons to consider the feasibility of dumping Google’s components from my phone and finding open source replacements for them all.

Whilst Google deserves credit for making the base OS comparability easy to build for users of Google-approved devices, the fact that they’ve allowed vendors to get away with binary blobbing drivers everywhere and keeping key functionality proprietary (market, etc) is pretty bad.

Chasing down binary blobs in order to get a device to work as expected is much more reminiscent of days spend pirating software, not of a healthy open source project and it makes Android feel much more hacky and crappy than it should be.

And the fact that the open source build will work with so few of the phones on the market out-of-the-box is just appalling for an OS that’s called open source – I should be able to go pickup any Android phone in the market and be able to compile AOSP for it and have all the hardware supported, not just select models.

Part of this is the fault of device manufacturers, but IMHO, Google should have put down some restrictions in the use of the Android trademark, so that a device could only actually be an “Android” device if it was fully open source and featured an unlocked bootloader.

I’d even accept a compromise where binary blobs are needed for specific hardware, as long as the blob wrapper can be compiled against different kernels and is free to redistribute (aka firmware style), so that I could buy a phone running Android 2 and happily go and build Android 4 for it and expect it to work.

Overall, I’m happy that I could build a functional image for my Nexus S without too much pain, but disappointed that so much of the feature set we are used to with Android isn’t actually open.