Author Archives: Jethro Carr

Why I hate URL shorteners

I’ve used Awstats for years as my website statistics/reporting program of choice – it’s trivial to setup, reliable and works with Apache log files and requires no modification to the website or usage of remote tools (like with Google Analytics).

One of the handy features is the “Links from an external page” display, which is a great way of finding out where sudden bursts of hits are coming from, such as news posts mentioning your website or other bloggers linking back.

Sadly over the past couple of years it’s getting less useful thanks to the horrible wonder that is URL shortening.

URL shorteners have always been a controversial service – whilst they can be a useful way of making some of the internet’s more horrible website URLS usable, they cause a number of long term issues:

Centralisation – The internet works best when decentralised, but URL shortening makes a large number of links dependent on a few particular organisations who may or not be around in the future. There’s already been a number of link shortening companies who have closed down killing large numbers of links and there will undoubtedly be more in the future.
Link Hiding – Short URLs are a great way to send someone a link and have them open it without realising what content they’re actually about to open. It could be as innocent as a prank for a friend or as bad as malicious malware or scamming websites.
Performance – It takes an extra DNS query (or several) to lookup the short URL servers before the actual destination can be looked up. This sounds like a minor issue, but it can add up when on high latency connections (eg mobile) or when connecting to international content on NZ’s wonderful internet and can add up to a number of seconds sometimes.
Privacy – a third party can collate large amounts of information about an individual’s browsing history if they have a popular enough URL shortening service.

Of course URL shortening isn’t entirely evil, there’s a few valid use cases where they are acceptable or at least forgivable:

Printed materials with URLs on them for manual entry. Nobody likes typing more than they need to, that’s understandable.
Quickly sending temporary links to people via IM or email where the full URL breaks due to the client application’s inability to phrase the URL correctly.

Anything other than the above is inexcusable, computers are great at hiding the complexities of large bits of information, there’s no need for your blog, social network or application to use short URLs where there is no human entry factor involved.

Twitter is particularly guilty at abusing short URLs – part of this was originally historic, but when Twitter had the opportunity to fix, they chose to instead contribute further towards the problem.

Back in the early days of Twitter, there was no native URL handling, so in order to fit many links into the maximum tweet size of 140chars, users would use a URL shortener such as the classic tinyurl.com or more recent arrivals such as bit.ly to keep the URL lengths as small as possible.

Twitter later decided to implement their own URL shortening service called t.co and now enforce the re-writing of all URLs posted via Twitter to use t.co links, in a semi-transparent fashion where some/all of the original URL will be shown in the tweet, but the actual hyperlink will always go through t.co.

This change offers some advantage to users in that they were no longer dependent on external providers closing down and breaking all their links, as well as having some security advantages in that Twitter maintain lists of bad URLs (URLs they consider to serve malware or other unwanted content) to help stop the spread of dodgy content.

But it also gave Twitter the ability to track click data to figure out which links users were clicking on, I imagine this information would be highly valuable to advertisers. (Google do a very similar thing with the Google results web pages, where all clicks are first directed through a Google server to track what results users select, before the user is delivered to the requested page).

The now mandatory use of URL shorteners on Twitter has lead to a situation where it’s no longer easy to track which tweets or even, what tweeters, are leading to the source of your hits.

Even more confusingly, the handling of referred URLs is inconsistent depending on the browser/client following the link. The vast majority will log as the short URL version, but some will be smart enough to provide the referred URL *before* the referring took place.

RFC 2616 doesn’t touch on how shortened URLs should be handled when referring and leaves the issue of how 301 redirects should have their referrers handled up to the implementers decision. And their are valid arguments for using the original page vs the short URL as the referrer.

For example, for this tweet I have about 9 visits via http://twitter.com/jethrocarr/status/170112859685126145 and 29 visits via http://t.co/0RJteq3r, which throws out hit-count based ordering of the results:

Got to love Twitter & shortened URLs - most of these relate to tweets, but to which tweets? No easy way to track back.

A much better solution, would have been for twitter to display shortened versions of URLs in the tweet text to meet the 140 char limit, but the actual link href record featuring the full URL – for example, a tweet could have “jethrocarr.com/i-like-…..” as the link text to fit within 140 chars, but the actual href record would be the full “jethrocarr.com/i-like-cake” URL.

Whilst tweets are known as being 140 chars, there’s actually far more information than that sorted about each tweet: location co-ordinates, full URL information, date, time and more, so there is no excuse for Twitter to not be able to retain that URL data – of course, that information has value for them for advertising and tracking purposes, so I wouldn’t expect it to ever go away.

(As a side note, there’s an excellent write up on ReadWriteWeb about the structure of a tweet and associated information)

Over all, shortened URLs are just a pain for dealing with and it would be far better if people avoided them as much as possible, essentially if you’re using a short URL and it’s not because a user will be manually typing out content, then you’re doing it wrong.

Also keep in mind that many sites have their own shortish URL variations. For example, this article can be accessed via both date/name and ID number:

https://www.jethrocarr.com/2012/02/26/why-i-hate-url-shorteners
https://www.jethrocarr.com/?p=1453

Many people also run their own private shorteners, quite common with popular sites such as news websites wanting to retain control of the link process and is a much better idea if you plan to have lots of short URLs for your website for a valid reason.

Virtualbox Awesomeness

3 Replies

Work recently upgraded us to the latest MS Office edition for our platform. Most of our staff run MacOS, but we have a handful of Windows users and one dedicated Linux user (guess who?) who received MS Office 2010 for Windows.

I’ve been using MS Office 2007 under Wine for several years, it was never perfect, but about 90% of the functionality worked with some exceptions such as PDF export and certain UI and performance artifacts.

With the 2010 upgrade I decided to instead switch to using Windows under a VM on my laptop to avoid any headaches and to fix the missing features and performance issues experienced running Office under Wine.

Whilst I’m a fan of Xen and KVM, they aren’t so well suited for desktop virtualisation as they’re designed more for server environments and don’t offer some of the more desktop focused features such as seamless integration, video acceleration and easy point & click management interfaces.

Instead I went with VirtualBox thanks to it being mostly open source (open source with exception for a few extensions for USB 2.0 forwarding and network boot) and with a pretty good reputation as a decent VM application.

It also has some of the user-friendly desktop features you’d expect such as being able to forward USB hardware through to guest, mounting any folder on the host as a network share (without needing to setup samba) and 2D/3D video acceleration.

But the real killer feature for me was the seamless windows feature, which allows me to boot the virtual windows desktop and Windows applications alongside my Linux applications smoothly and without the nastiness of an RDP window.

Windows & Linux application windows running together concurrently.

Sadly it’s not quite good enough for you to be able to run the latest Windows games in as the 3D acceleration is quite basic, but it’s magnificent for just about any other non-multimedia application.

The only glitch I found, is that if you have dual screens, you can only run the windows session on one screen at a time, although virtualbox does allow moving the session between monitors whilst running so it’s not too big a deal.

The other annoying issue I had with virtualbox is that it uses image files for storing the guest VMs and it doesn’t appear possible to get it to use an LVM volume instead – so in my case, I waste a bit of space and performance for unnecessary filesystem formatting to store the Windows VM. I guess this is a feature that only a small subset of users would want so it’s not particularly high priority for them to add it.

I’m running Win7 with 2 virtual cores and 1GB of RAM on top of a host with an Intel Core i5 CPU (with hardware virtualisation enabled), 8GB RAM and a Intel 320 series SSD and it’s pretty damn snappy.

As a side note, the seemless window integration also works for Linux-based guests, so you could also do the same ontop of a Windows host, or even Linux-on-Linux if desired.

Johnsonville Train

Die Flash, Die!

4 Replies

“I hated flash whilst it was still cool!” — Jethro Carr, Internet Hipster

Adobe Flash has to be one of the more polarizing internet technologies out there, people either love it or hate it, but either way, it’s difficult to avoid. It’s used as the default for playing youtube videos, many online browser games, banner adds, “smart” uploaders and a large number of adult websites.

It’s also used for some important systems as well – Air New Zealand make heavy use of it for their Airports membership page (infact it’s not possible to login unless you have flash), which is extremely poor from a large company that should know better, along with a few too many enterprise web applications I’ve come across.

Whilst Flash has had a reputation for poor performance, CPU eating and battery-life killing, these are all implementation faults – the primary issue with Flash has always been that it’s a proprietary application and a proprietary standard.

If Adobe had simply allows Flash to become an open standard and open sourced the flash player, many of the technical issues with it would be resolved by the developer community, and it would become more ubiquitous with ports to other platforms that Adobe might consider “too small” to worth spending developer time with.

Adobe didn’t even release specifications and allow free licensing until 2009 when they kicked off the Open Screen Project and released the specification – but it’s a big catchup game to play for other applications to fully implement the specification needed to support flash applications. And the flash player itself is still fully proprietary, if Adobe doesn’t want to support a platform or a browser, you’re effectively screwed.

Open source projects like Gnash are slowly catching up, when I tried it recently it was good enough to allow me to play Youtube videos and some other flash features, but would fail on more complex applications such as Air New Zealand’s abomination of a website, so depending on your needs, you may still be chained to it.

Flash on Linux has always had a particularly rocky history – historically Adobe made a plugin available but only supported the i386 platform, requiring many years of the use of 32 to 64bit wrapper libraries in order to run Flash on modern 64bit Linux systems, leading to all sorts of wonderful performance, memory and audio issues.

A 64-bit alpha plugin emerged relatively recently and Adobe now supports 64-bit Linux as part of their official downloads, but other platforms such as PPC, MIPS and ARM are still unsupported – an issue which becomes more and more apparent as vendors release ARM based smart-phones and tablets and are unable to install flash player on them.

Adobe has now announced that they will be dropping support for Flash on Linux for anything but Google’s Chrome browser, which has it’s own special build in flash binaries – I suspect this will mean that it won’t extend to supporting the open source build of Chrome (called Chromium) which currently excludes the Flash support.

Of course, for other browser users like myself (eg Firefox), this decision is short sighted and very frustrating – a text book example of the problems with relying on proprietary software and standards.

Thankfully Adobe did at least realise that this decision is going to result in a lot of users sticking with the final 11.2 version on Linux and is promising to support 11.2 with security updates for another 5 years, so at least we won’t have thousands of users running around with vulnerable flash players – Flash Player does have a reputation for security holes after all.

On the positive side, Flash is dying.

Adobe has already announced plans to stop supporting mobile platforms like Android in favor of Adobe Air, although Adobe Air sounds like they’re making the mistakes of Flash all over again, unless they allow fully HTML5 based Air applications to run without need for a browser plugin in future.

Apple has always refused to support Flash on the iOS platform (iphone/ipad) and recently stopped shipping Flash with MacOS on Macbook Air by default. (in a hilariously ironic statement, Apple criticized Flash for being a proprietary locked down platform, whilst happily ruling the iOS platform and App store with an iron fist).

HTML5 along with Javascript is quickly securing it’s place as the web platform of choice for rich UI web application developers and I expect we’ll see more and more tools and frameworks to make working with these technologies easier.

You can even watch Youtube videos in HTML5 if you have a capable browser (recent versions of Chrome or Firefox will work) under their HTML5 trial.

Hopefully projects like Gnash are able to complete their implementation of Flash to a sufficient level to support legacy websites and applications, although by the time this happens, it may be that we won’t need it any more.

If Adobe had just open sourced Flash Player and the standards years ago, maybe this wouldn’t have been the case and we’d all be running stable open Flash implementations already, Adobe only has itself to blame for Flash’s demise.

But they won’t see any tears from me.

Why I hate DSL

2 Replies

mmm latency, delicious delicious latency

I’m living around 8km from the center of Auckland, New Zealand’s largest city, with only 8 mbits down and 0.84 mbits upstream in a very modern building with (presumably) good wiring installed. :'(

The above is what happens to your domestic latency when your server’s cronjobs decide to push 1.8GB of new RPMs up to a public repo, causing the performance to any other hosts to slow to a grinding halt. :-/

On the plus side it did make me aware of a fault in my server setup – one of my VMs was incorrectly set to use my secondary LDAP server as it’s primary authentication source, meaning it called back across the VPN over this DSL connection, so when this performance hit occurred, the server started having weird “hangs” due to processes blocking whilst waiting for authentication attempts to complete.

The sooner we can shift off DSL the better – there’s the possibility that my area might be covered with VDSL, but since I don’t think we’ll be here for too long, I’m not going to go to the effort to look at other access methods.

But I will make sure I carefully consider where UFB is getting laid when I come to buy property….

Makara Walk

2 Replies

Whilst in Wellington for a weekend a few weeks ago, I went on a walk around Makara with my good friend Tom, only I got a little too distracted to blog it until now.

Makara is a very small rural community near the coastline behind Wellington, only about 30mins drive from the Wellington CBD. Whilst you wouldn’t go there to visit the thriving shopping district or cafe scene (ha!), it is an interesting place for some walks up large hills with amazing views and a slight risk of deadly cliff drops or heart attacks from climbing the steep slopes.

Like most of Wellington, it’s a stony beach so you wouldn’t go there for the soft sand, warm swims and attractive sunbathers, but it does offer a bleakness that is strangely attractive.

No hot sunbathers? Feeling very ripped off.

We took the Maraka Walkway which starts off going steeply uphill through some quite undefined tracks (hint: follow the flattened grass paths), before going up to old WW2 gun emplacements, wandering through the windmills and then back along the coastline.

If one desired, there are longer paths that can be be taken right through the wind farms and into other bays. In theory, if you went far enough along the coast, you would end up going around the southern most point of Wellington and reconnecting to civilization at Red Rocks, or even going up to the Brooklyn wind turbine.

The map doesn't quite give a warning to the steepness, also they weren't kidding with the recommendation for some light walking boots, some of the areas are a bit too muddy and rough for sneakers.

Erect Rocks.

Oh mercieless sea, so strong yet so frail as you fall upon these rocks of land. Ponder thee Tom shall.

Unsure what this building is - it's in the right area on the map for the Pa site that we couldn't see any other trace of, but it's less than 100 yrs old as it's clearly built of cement and steel and not a Pa...

Climbing rapidly...

Looking out from a cliff towards the south island

The path winds along these pretty steep hills, wouldn't want to go for a tumble down one.

Based on the number of these spider nests around, I would *not* want to be doing this walk whenever it is that these things hatch :-/

Unfit Jethro is unfit and generally just a bit hot.

Whilst New Zealand never had enemy troops landing, we did have the odd axis vessel in our waters and a few shipping casualties and flyovers by German and Japanese craft.

During the war a number of gun emplacements were built to fend off invasion and there were American troops stationed in the country, although I don’t know why the Japanese and German forces wasted time/effort with New Zealand, it’s too far away from any action and Australia would be a much better target.

After the war these were mostly dismantled and the guns taken away, although the concrete emplacements were left as-is.

WW2 gun emplacements, sadly the guns are long gone, all that's left are decaying concrete structures, now fenced off due to their unstable nature.

I'm defending the windmills!

Fort Opau Gun Emplacements

Friendly sheep! (Makara Farm is spread over these hills)

The other man-made attraction in this area is the modern Makara windfarm build in 2007-2009 and has 62 turbines producing up to 2.3MW each as Project West Wind (the budget did not extend to original project naming it seems).

I love wind turbines, they look pretty over a countryside, are a clean and effective form of generating power and combined with hydro, could allow New Zealand’s energy production to become completely sustainable.

This particular wind farm had history dating back to 1995 and a number of fights with a group known as the Makara Guardians, made up of a lot of the residents of the area to fight the turbine installation.

This particular selfishness of groups like Makara Guardians really gets to me, New Zealand needs energy and clean renewable sources such as wind and hydro are the best way for us to get them, yet they’re opposed thanks to the “not in our backyard” mentality and quoting invalid pseudoscience to try and justify their arrogance. Would they prefer a coal plant in Makara? Or no power?

Thankfully the environment court ruled in favor of allowing the project to go forwards so the farm was built and from what I see, with minimal impact to the landscape and environment – the sites of each turbine are tidy and minimal and without unpleasant overhead wires between each turbine.

They are also amazingly quiet, it was a light breezy day day, but we couldn’t hear anything until right up close to them and even then they were still far quieter than on-street car noise or other city sounds.

Pretty wind turbines!

These things are big!

REALLY BIG!

No geek could resist a functional diagram and pictures of construction! :-D

After geeking out at Turbines, we headed down to Ohau Bay and then along the coastline all the way back to Makara.

I will note that whilst the map earlier showed this as a path, it’s really not…. once on the beach, there’s a very minimal pathway that’s mostly covered by rubble from the hills or driftwood from the beach, and in a few places actually runs out entirely and the “path” ends up being climbing over rocks whilst the waves crash around you. I have no idea what happens at high tide, it might not be possible to cross some areas at all.

Whilst I got away with sneakers, it was really a case of barely… I’d highly recommend getting some decent walking/tramping boots if attempting this walk, particularly if there’s a chance that the weather might not be the greatest.

Swampy area

Of course, I walked right though it. :-/

Yo dawg, we heard you liked driftwood, so we put some driftwood on your driftwood!

The coastline "path" :-/

Pretty in a bleak way.

Apparently this counts as a "path" :-/

Wouldn't want to be out here during a storm....

Home stretch!

Over all it was a great trip and certainly a bit more challenging walk than the usual well paved ones found around Wellington. It’s pretty exposed so you want some good weather, but if it’s a bit overcast it helps since there’s little shelter from the sun.

Sea Shepherd in Wellington

3 Replies

Whilst wandering along Wellington harbor this week I came across the MY Bob Barker berthed, one of the things I love about Wellington is finding random bits of awesomeness like that.

This is actually the second Sea Shepherd ship I’ve seen IRL, in early 2010 they had the MY Steve Irwin in Wellington and I managed to get some pictures of that then too.

Sea Shepherd is an interesting organization with a background showing that they aren’t afraid to take more extreme actions than just protesting, with past history including ramming whaling ships and contributing towards the sinking of several – reading the fleet history gives a good idea of some of their past exploits.

I can’t stand whalers, they’re ignorantly and blatently killing extremely intelligent creatures and in the case of Japan, they don’t even have the courage to call the hunt and slaughter what it is, instead calling it “research” and miss-using loopholes in multinational treaties.

The politicians, companies and people involved in whale hunting do nothing but bring shame and condemnation on their countries and deserve to end up on the bottom of the ocean, the sooner the better.

Wellington 13th to 17th

1 Reply

I’ll be in Wellington from Monday the 13th till Friday the 17th of Feb on various customer work projects that can’t be done via a remote shell.

Due to the nature of the work I’m not sure what evenings I’ll be free, but probably be free for dinner on Monday evening and may have other dates available later in the week.

If you’re around and would like to catch up, chuck me an email or comment and we’ll make a time. :-)

Takapuna Beach Wanders

Mozilla Firefox Sync Awesomeness

1 Reply

When I recently rebuilt my laptop, I switched back from Google’s Chromium browser to instead use Mozilla Firefox. Whilst I really liked Chromium, there were a few specific reasons which push me more towards Firefox:

I trust the Mozilla Foundation to uphold my desire for producing a great browser more than I trust Google. This isn’t because I believe Google to be evil, but rather that Google’s motivations are to sell advertising, whereas Mozilla’s is to produce a great browser.
The performance issues that pushed me away from Firefox 3 have largely been rectified, reducing the benefit of Chromium.
Standardising my systems on Firefox, means I can make use of Firefox Sync

This last one is of particular interest to me – Firefox Sync is a feature which allows all your Firefox instances to synchronise data between themselves, using a freely provided public server or your own server.

I think this is one of the most compelling user feature improvements since they released version 1.5 IMHO – the ability to synchronise my bookmarks, credentials, history and best of all, open tabs lists, allows me to effortlessly have multiple devices for browsing without dealing with the frustrating issue of the bookmark or saved password being on the other machine.

For example, I setup a Firefox instance on my Linux laptop, Windows VM and my phone and could see all the tabs between them:

Tabs from Linux laptop's POV

Tabs from Windows VM's POV

Tabs from Firefox on Android's POV

I’ve found this tab synchronization to be one of the most useful features, it makes it so much easier to find an article I was in the process of reading to finish whilst I have free time when traveling.

If you’re one of those people with multiple devices (desktops, laptops, work machine, personal machine, tablets, phones, etc) it makes life so much simpler. And as long as you have Firefox 4+, it ships as a native feature.

For details about using Firefox Sync and how it works see Mozilla’s details on the service and/or read the getting started with Firefox Sync FAQ.

As you probably know if you ever read my blog, I’m pretty keen on using an entirely open source stack – and so Firefox Sync is naturally fully open source, including both the client (Firefox) and the server components.

This lack of an open source server kills Chromium for me, since the Chrome sync relies on your Google Account and their servers. :-(

Using Mozilla’s open source sync server isn’t as easy as I’d like it to be… building a working server from their source code and limited documentation is a bit of a mission, particularly when some documentation is outdated and doesn’t apply to the latest commit, or when troubleshooting documentation barely exists.

However I’ve managed to successfully package RPMs against CentOS 5 for sync server and dependencies and am currently running further tests before I release them. Ideally I’ll make some time to build them against CentOS 6 as well (done them against CentOS 5 first, since that’s my current production host OS of choice – also the hardest since the version of python it ships with is too old).

With support for SQLite, LDAP, MySQL and Memcache it’s quite flexible and designed to scale to the huge user volumes that Mozilla undoubtedly have – I’ve been running tests with SQLite, but will be having a play with MySQL and OpenLDAP over the next few days as it would be nice to align it to my existing LDAP server.

Expect another blog post later this week with details on how to obtain the RPM packages, along with instructions on setting up your own sync server. It only took me about 3 days full effort of packaging weird python dependencies and getting a working set of configuration and spec files for the sync server to make this stuff work, so hopefully someone is actually interested!