Tag Archives: debian

Easy APT repo in S3

When running a number of Ubuntu or Debian servers, it can be extremely useful to have a custom APT repo for uploading your own packages, or third party packages that lack their own good repositories to subscribe to.

I recently found a nice Ruby utility called deb-s3 which allows easy uploading of dpkg files into an S3-hosted APT repository. It’s much easier than messing around with tools like reprepro and having to s3 cp or sync files up from a local disk into S3.

One main warning: This will create a *public* repo by default since it works out-of-the-box with the stock OS and (in my case) all the packages I’m serving are public open source programs that don’t need to be secured. If you want a *private* repo, you will need to use apt-transport-s3 to support authenticating with S3 to download files and configure deb-s3 for private upload.

Install like any other Ruby Gem:

gem install deb-s3

Adding packages is easy. First make sure your aws-cli is working OK and an S3 bucket has been created, then upload with:

deb-s3 upload \
--bucket example \
--codename codename \
--preserve-versions \
mypackage.deb

You can then add the repo to a Ubuntu or Debian server with:

# We trust HTTPS rather than GPG for this repo - but you can config
# GPG signing if you prefer.
cat > /etc/apt/sources.list.d/myrepo.list << EOF
deb [trusted=yes] https://example.s3.amazonaws.com codename main
EOF

# and ensure you update the package info on the server
apt-get update

Alternatively, here’s an example of how to add the repo with Puppet:

apt::source { 'myrepo':
 comment        => 'This is our own APT repo',
 location       => 'https://example.s3.amazonaws.com',
 release        => $::os["distro"]["codename"],
 repos          => 'main',
 allow_unsigned => true, # We don't GPG sign, HTTPS only
 notify_update  => true, # triggers apt-get update
}

Upcycling 32-bit Mac Minis

13 Replies

The first generation Intel Apple Mac Mini (Macmini1,1) has a special place as the best bang-for-buck system that I’ve ever purchased.

Purchased for around $1k NZD in 2006, it did a stint as a much more sleep-friendly server back after I started my first job and was living at my parents house. It then went on to become my primary desktop for a couple of years in conjunction with my laptop. And finally it transitioned into a media centre and spent a number of years driving the TV and handling long running downloads. It even survived getting sent over to Sydney and running non-stop in the hot blazing hell inside my apartment there.

My long term relationship on the left and a more recent stray obtained second hand.

My long term relationship on the left and a more recent stray I obtained. Clearly mine takes after it’s owner and hasn’t seen the sun much.

Having now reached it’s 10th birthday, it’s started to show it’s age. Whilst it handles 720p content without an issue, it’s now hit and miss whether 1080p H264 content will work without unacceptable jitter.

It’s previously undergone a few upgrades. I bumped it from the original 512MB RAM to 2GB (the max) years ago and it’s had it’s 60GB hard drive replaced with a more modern 500GB model. But neither of these will help much with the video decoding performance.

Given we had recently obtained something that the people at Samsung consider a “Smart” TV, I decided to replace the Mac Mini with the Plex client running natively on the TV and recycle the Mac Mini into a new role as a small server to potentially replace a much more power hungry AMD Phenom II system that performs somewhat basic storage and network tasks.

Unfortunately this isn’t as simple as it sounds. The first gen Intel Mac Minis arrived on the scene just a bit too soon for 64-bit CPUs and so are packing the original Intel Core Solo or Intel Core Duo (1 or 2 cores respectively) which aren’t clocked particularly high and are only 32bit capable.

Whilst GNU/Linux *can* run on this, supported versions of MacOS X certainly can’t. The last MacOS version supported on these devices is Mac OS X 10.6.8 “Snow Leopard” 32-bit and the majority of app developers for MacOS have decided to set their minimum supported platform at 64-bit MacOS X 10.7.5 “Lion” so they can drop the old 32-bit stuff – this includes the popular Chrome browser which now only provides 64-built builds. Basically OS X Snow Leopard is the Win XP of the MacOS world.

Even running 32-bit GNU/Linux can be an exercise in frustration. Some distributions now only ship 64-bit builds and proprietary software vendors don’t always bother releasing 32-bit builds of their apps limiting what you can run on them.

On the plus side, this earlier generation of Apple machines was before Apple decided to start soldering everything together which means not only can you replace the RAM, storage, drives, WiFi card, you can also replace the CPU itself since it’s socketed!

I found a great writeup of the process at iFixit which covers the process of replacing the CPU with a newer model.

Essentially you can replace the CPUs in the Macmini1,1 (2006) or Macmini2,1 (2007) models with any chip compatible with Intel Socket M, the highest spec model available being the Intel Core 2 Duo 2.33 Ghz T7600.

At ~$60NZD for the T7600, it was a bit more than I wanted to spend for a decade old CPU. However moving down slightly to the T7400, the second hand price drops to around ~$20NZD per CPU with international shipping included. And at 2.177Ghz it’s no slouch, especially when compared to the original 1.5Ghz single core CPU.

It took a while to get here, I used this seller after the first seller never delivered the item and refunded me when asked about it. One of my CPUs also arrived with a bent pin, so there was some rather cold sweat moments straightening the tiny pin with a screw driver. But I guess this is what you get for buying decade old CPUs from a mysterious internet trader.

I was surprised at the lack of dust inside the unit given it’s long life, even the fan duct was remarkably dust-free.

The replacement is a bit of a pain, you have to strip the Mac Mini right down and take the motherboard out, but it’s not the hardest upgrade I’ve ever had to do – dealing with cheap $100 cut-your-hand-open PC cases were much nastier than the well designed internals of the Mac. The only real tricky bit is the addition and removal of the heatsink which worked best with a second person helping remove the plastic pegs.

I did it using a regular putty knife, needle-nose pliers, phillips & flat head screw drivers and one Torx screw driver to deal with a single T10 screw that differs from the rest of the ones in the unit.

Recommend testing this things *before* putting the main case back together, they’re a pain to open back up if it doesn’t work first run.

The end result is an upgrade from a 1.5 Ghz single core 32bit CPU to 2.17 Ghz dual core 64bit CPU – whilst it won’t hold much to a modern i7, it will certainly be able to crunch video and server tasks quite happily.

The next problem was getting an OS on there.

This CPU upgrade opens up new options for MacOS fans, if you hack the installer a bit you can get MacOS X 10.7.5 “Lion” on there which gives you a 64-bit OS that can still run much of the current software that’s available. You can’t go past Lion however, since the support for the Intel GMA 950 GPU was dropped in later versions of MacOS.

Given I want them to run as servers, GNU/Linux is the only logical choice. The only issue was booting it… it seems they don’t support booting from USB flash drives.

These Mac Minis really did fall into a generational gap. Modern enough to have EFI and no legacy ports, yet old enough to be 32-bit and lack support for booting from USB. I wasn’t even sure if I would even be able to boot 64-bit Linux with a 32-bit EFI…

Given it doesn’t boot from USB and I didn’t have any firewire devices lying around to try booting from, I fell back to the joys of optical media. This was harder than it sounds given I don’t have any media and barely any working drives, but my colleague thankfully dug up a couple old CD-R for me.

“Daddy are those shiny things floppy disks?”

I also quickly remembered why we all moved on from optical media. My first burn appeared to succeed but crashed trying to load the bootloader. And then refused to eject. Actually, it’s still refusing to eject, so there’s a Debian 8 installer that might just be stuck in there until it’s dying days… The other unit’s optical drive didn’t even work at all, so I couldn’t even do the pain of swapping around hardware to get a working combination.

Having exhausted the optional of a old-school CD-based GNU/Linux install, I started digging into ways to boot from another partition on the machine’s hard drive and found a project called rEFInd.

This awesome software is an alternative boot manager for EFI. It differs from a boot loader slightly, in a traditional BIOS -> Boot Loader -> OS world, rEFInd is equivalent to a custom BIOS offering better boot functionality than the OEM vendor.

It works by installing itself into a small FAT partition that lives on the hard disk – it’s probably the easiest low-level tool I’ve ever installed – download, unzip, and run the installer from either MacOS or Linux.

Disturbingly easy from the existing OS X installation

Once installed, rEFInd kicks in at boot and offers the ability to boot from USB flash drives, in addition to the hard drive itself!

The USB flash installer has been detected as “Legacy OS from whole disk volume”.

Yusss, Debian installer booted from USB via rEFInd!

A typical Debian installation followed, only thing I was careful about was not to delete the 209.7MB FAT filesystem used by EFI – I figured I didn’t want to find out what deleting that would mean on a box that was hard enough to boot as it is…

The small < 1MB free space between the partitions here irks me so much, I blame MacOS for aligning the partitions weirdly.

Once installed. rEFInd detected Linux as the OS installed on the hard drive and booted into GRUB and from there the usual Linux boot process works fine.

Launch the penguins!

This spec sheet violates the manafacturer's warranty!

Final result -2GB RAM, 64bit CPU, delicious delicious GNU/Linux x86_64

I can confirm that both 32bit and 64bit Debian works nicely on this box (I installed 32-bit first by mistake) – so even without doing the CPU upgrade, if you want to get a bit more life out of these early unsupported Mac Minis, they’d happily run a 32-bit Debian desktop so you can enjoy wonders like a properly patched browser and operating system.

Not all other distributions will work – Ubuntu for example don’t include EFI support on their 32-bit installer which will probably cause some headaches. You should be OK with any of the major 64-bit distributions since they tend to always support EFI.

The final joy I ran into is that when I set up the Mac Mini as a headless box, it didn’t boot… it just turned on and never appeared on the network.

Seems that the Mac Minis (even the later unibody generation) have some genius firmware that disables the GPU hardware if no screen is attached, which then messes up most operating systems on it.

The easy fix, is to hack together a fake VGA load by connecting a 100Ω resister between pins 2 and 7 of a DVI-to-VGA adaptor (such as the one that ships with the Mac Mini).

I need to make a tidier/better version of this, but it works!

No idea what engineer thought this was a good feature, but thankfully it’s an easy and cheap fix, especially since I have a box littered with these now-useless adaptors.

The end result is that I now have 2x 64-bit first gen Mac Minis running Debian GNU/Linux for a cost of around $20NZD and some time dismantling/reassembling them.

I’d recommend these small Mac Minis for server purposes, but the NZ second hand prices are still a bit too expensive for their age to buy specifically for this… Once they start going below $100 they’d make reasonable alternatives to something like the Intel NUC or Raspberry Pi for small serving tasks.

The older units aren’t necessarily problem free either. Whilst the build quality is excellent, after 10 years things don’t always work right. Both of my optical drives no longer function properly and one of the Mac Minis has a faulty RAM slot, limiting it to 1GB instead of the usual 2GB.

And of course at 10 years whom knows how much longer they’ll run for – but it’s been a good run so far, so here’s to another 10 years (hopefully)! The real limiting factor is going to be the 1GB/2GB RAM long term.

Updated Repositories

2 Replies

I’ve gone and updated my GNU/Linux repositories with a new home page – some of you may have been using this under my previous Amberdms branding, but it’s more appropriate that it be done under my own name these days and have it’s own special subdomain.

I want to unify the branding of a bit more of the stuff I have out there on the internet and also make sure I’m exposing it in a way that makes it easy for people to find and use, so I’m going through a process of improving site templates, linking between places and improving documentation/wording with the perspective of viewing as an outside user.

CSS3 shinyness! And it even mostly works in IE.

Been playing with new HTML5/CSS3 functionality for this site, have to say, it’s pretty awesome.

You can check out the new page at repos.jethrocarr.com, I’ve tried to make it as easy as possible to add my repositories to your servers -I’ll be refining this a little more in coming weeks, such as adding a decent package search function to the site to make it easier to grab some of the goodies hidden away in distribution directories.

I’m currently providing packages for RHEL & clones, Debian and Ubuntu. Whilst my RHEL repos are quite sizable now, the Debian & Ubuntu repositories are much sparser, so I’m going to make an effort to bring them to a level where they at least have all my public software (see projects.jethrocarr.com) available as well tested packages for current Debian Stable and Ubuntu LTS releases.

There’s some older stuff archived on the server if you go hunting as well, such as Fedora and ancient RHEL version packages, but I’m keeping them in the background for archival purposes only.

And yes, all packages are signed with my Amberdms/Jethro Carr GPG signing key. You should never be using any repositories without GPG signed packages, since they’re ideal attack vectors to use to install malicious content with a man-in-the-middle attack otherwise.

Debian Testing with Cinnamon

7 Replies

I’ve been running Debian Stable on my laptop for about 10 months for a number of reasons, but in particular as a way of staying away from GNOME 3 for a while longer.

GNOME 3 is one of those divisive topics in the Linux community, people tend to either love it or hate it – for me personally I find the changes it’s introduced impact my workflow negatively, however if I was less of a power user or running Linux on a tablet, I can see the appeal of the way GNOME 3 is designed.

Since GNOME 3 was released, there have been a few new options that have arisen for users craving the more traditional desktop environment offered – two of the popular options are Cinnamon and MATE.

MATE is a fork of GNOME 2, so duplicates all the old libraries and applications, where as Cinnamon is an alternative GNOME Shell, which means that it uses the GNOME 3 libraries and applications.

I’m actually a fan of a lot of the software made by the GNOME project, so I decided to go down the Cinnamon path as it would give me useful features from GNOME 3 such as the latest widgets for bluetooth, audio, power management and lock screens, whilst still providing the traditional window management and menus that I like.

As I was currently on Debian Stable, I upgraded to Debian Testing which provided the required GNOME 3 packages, and then installed Cinnamon from source – pretty easy since there’s only two packages and as they’ve already packaged for Debian, just a dpkg-buildpackage to get installable packages for my laptop.

So far I’m pretty happy with it, I’m able to retain my top & bottom menu bar setup and all my favorite GNOME applets and tray features, but also take advantages of a few nice UI enhancements that Cinnamon has added.

All the traditional features we know and love.

One of the most important features for me was a functional workspace system that allows me to setup my 8 different workspaces that I use for each task. Cinnamon *mostly* delivers on this – it correctly handles CTL+ALT+LEFT/RIGHT to switch between workspaces, it provides a taskbar workspace switcher applet and it lets me set whatever number of workspaces I want to have.

Unfortunately it does seem to have a bug/limitation where the workspace switcher doesn’t display mini icons showing what windows are open on which workspace, something I often use for going “which workspace did I open project blah on?”. I also found that I had to first add the 8 workspaces I wanted by using CTL+ALT+UP and clicking the + icon, otherwise it defaulted to the annoying dynamic “create more workspaces as you need them” behavior.

On the plus side, it does offer up a few shinier features such as the graphical workspace switcher that can be opened with CTL+ALT+UP and the window browser which can be opened with CTL+ATL+DOWN.

You can never have too many workspaces! If you’re similarly anal-retentive as me you can go and name each workspace as well.

There’s also a few handy new applets that may appeal to some, such as the multi-workspace window list, allowing you to select any open window across any workspace.

Window applet dropdown, with Nautilus file manager off to the left.

I use Rhythmbox for music playback – I’m not a huge fan of the application, mostly since it doesn’t cope well with playing content off network shares over WAN links, but it does have a nice simple UI and good integration into Cinnamon:

Break out the tweed jackets and moleskins, you can play your folk rock in glorious GTK-3 graphics.

The standard Cinnamon theme is pretty decent, but I do find it has an overabundance of gray, something that is quite noticeable when using a window heavy application such as Evolution.

Didn’t you get the memo? Gray is in this year!

Of course there are a lot of other themes available so if the grayness gets to you, there are other options. You also have the usual options to change the window border styles, it’s something I might do personally since I’m finding that the chunky window headings are wasting a bit of my laptop’s very limited screen real estate.

Overall I’m pretty happy with Cinnamon and plan to keep using it for the foreseeable future on this laptop – if you’re unhappy with GNOME 3 and preferred the older environment, I recommend taking a look at it.

I’ve been using it on a laptop with a pretty basic Intel GPU (using i810 driver) and had no issue with any of the accelerated graphics, everything feels pretty snappy – there is also a 2D Cinnamon option at login if your system won’t do 3D under any circumstance.

apt-get install debian

11 Replies

Early last year I wrote how I was concerned about the progress and future of the CentOS project and was considering other options.

As of today, I’ve now shifted my primary workstation (Lenovo X201i laptop) from the somewhat out of date Fedora 13 over to Debian Stable/Squeeze.

The main drives for this change were:

Fedora 13 was out-of-date and lacking security fixes, hence requirement to upgrade.
The logical replacement, Fedora 16, now ships with GNOME 3 which I found to be buggy and a regression to my work flow and requirements (not going into details here, but essentially issues with dual screens, workspaces and customization of toolbars).
Desire to seriously try Debian as a primary system with the purpose of evaluating it’s suitability as a replacement for my CentOS servers.
Requirement for a distribution that made major release upgrades feasible (Fedora can do version jumps, but not recommended, making it a tricky process to find time to do a laptop upgrade/rebuild).
Distribution standardisation across my server & workstation environments.
I needed a full reinstall in order to downsize from a 320GB HDD to a 120GB SSD.
Reliability – my laptop is my primary business machine, if it doesn’t work, I’m going to be living on instant noodles until it starts working. Or even worse, work will buy me a Macbook to use like everyone else. :-/

I chose Debian particularly, since it would be a fine option for replacing my CentOS servers in the future with long life support & stability, it’s large package selection and the fact that it’s committed to freedom and openness (as is Fedora also); all of which made it more attractive than Ubuntu for me, which feels much more desktop and fast release focused.

So far, I’m loving it – the distribution is solid, well built and developer friendly, and the package selection is pretty decent, not to mention apt being nice and snappy (although the SSD sure helps there ;-)

I’ve had a couple minor issues relating to my Lenovo hardware that I’ve been able to resolve and have gotten into building a few Debian packages in order to backport newer versions of programs like Firefox/Iceweasel.

From what I’ve observed with playing with Debian today is that’s a pretty awesome distribution, but not entirely as perfect as some of it’s users like to make out:

The installer is a bit dated – not due to the text installer (I fucking love text installers! \m/), but rather due to it’s lack of support for WPA/2 wifi access points and also the ability to allow the user to make broken systems without warning (eg no /boot partition when you don’t have enough coffee like me).
Debian is often credited for having all the packages under the sun in it – whilst almost true, I did note that a number of my more obscure package choices didn’t exist, sending me running for my compiler a few times.
It would be nice if stable backports tracked some of the common packages that users like updated on older systems – programs like web browsers, instant messengers and maybe even kernels for uncooperative hardware. A user could avoid this by using Debian testing, but there are valid reasons to use stable + some backports over using testing or unstable.
I think rpm has nicer command line options for directly working with installed or to-be-installed packages than dpkg. Having said that, some of this could be user familiarity/likes, so time will tell as I use it more.

Over all though, these are minor issues – I think it’s a fantastic distribution with so much working out of the box, applications appearing well tested and good online documentation and resources.

I’m currently running trials and comparisons of Debian with my CentOS hosts with a view for replacing my current CentOS 5 instances with Debian Stable instances over a phased migration period, as well as testing features like LDAP authentication and KVM, but it’s looking pretty damn good so far.

At this stage I’ve only used CentOS 6 as a KVM host platform and it seems unlikely I’ll end up deploying any CentOS 6 VMs with all the security update release slowness. With only a couple servers on CentOS 6 altogether, I’m pondering switching them over to Debian sooner rather than later to reduce maintenance efforts.

[FYI, this post isn’t intended as an attack/demands at the CentOS developers, rather it’s recognizing they’re a volunteer team and probably lacking resource – and I thank them for their efforts, but it appears long term it’s not a good option for my requirements.]

It does also raise questions about Red Hat’s RHEL future with engineers like myself – with Red Hat no longer offering a free-as-in-beer-with-no-support option and CentOS being too slow, more geeks like myself might move to Debian. And if we do so, when the next enterprise project comes along, will we be recommending RHEL or Debian?

Red Hat offers the advantage of commercial support, but for a company with their own engineers, Debian may be more appropriate and budget friendly.

CentOS, RHEL and future possibilities?

10 Replies

Those who know me will know that I’m a long term CentOS user – this actually started from my love of RHEL, back in my early Linux using days when I was running Red Hat 8.0.

Whilst it made financial sense for Red Hat to switch to making their product only available in binary form for their customers, at the same time I can’t help but feel this has damaged the appeal of Red Hat for geeks like myself – I’m no longer able to setup friends, family or customers without the funds for RHEL with a quality, enterprise-grade free (as in beer + freedom) distribution.

I do wonder if this contributes to reduced market awareness in the small business space and also whether it reduces the likeliness of geeks like myself promoting the software – after all, if I can’t run RHEL myself, I’m likely to look at other distributions and options and end up promoting those.

With the lack of a free Red Hat enterprise-grade distribution, there are only a couple options for wanting a Red Hat-style experience:

Fedora – the community developed distribution that forms the future base of RHEL, a fantastic distribution in it’s own right, but with only 12 months support per release, not suitable for server deployments.
CentOS – the community free re-spin of RHEL with their trademarks removed to make it legally redistributable.

I’ve been using CentOS heavily on my servers and Fedora on my workstations, however there are a number of security delays that are concerning me about CentOS which have been recently highlighted in an LWN article.

Essentially, the core problem is that the latest version of CentOS is still only 5.5, whilst Red Hat have had 5.6 out for some time, with numerous security updates in it that have yet to be released for CentOS…..

Having systems vulnerable to known exploits with no upstream patches is always a pretty serious concern to any system administrator…. this is leading me to re-think my usage of CentOS and to consider whether I should consider other platforms.

I’ve never been a huge fan of Debian in the past, but I’m considering giving it a more detailed look and try – Debian has the advantages of a strong community (like Fedora has) but without the limitation of a short support life – although then again, Debian’s releases and support spans are a little less rigid than Red Hat, which is somewhat annoying.

There’s a few server platforms that come to mind – Ubuntu LTS, Mint Linux, Debian, Open/SuSe or of course, Fedora.

The other option is that I could spin my own distribution – based on the number of custom RPMs I already build, rebuilding Red Hat’s update packages for my own needs wouldn’t be too hard, but I really don’t want to get caught up in distribution maintenance for the next 5 years plus it’s not suitable for customer deployments – so even if I decide that a custom built system is best for me, it still doesn’t solve the “what do I install for others?” question.

Maybe I need Fedora LTS – long term support for specific versions of Fedora – 3 or 5 years would be wonderful and meet the needs of server administrators.

This was tried once before, with the Fedora Legacy project, but it didn’t last long – possibly the goal of supporting *all* the releases was too much to reasonably handle, so an approach of selection even/odd number releases only might make it more feasible – I know that I’d certain be willing to contribute.

Anyway, this is a late night concerned system administrator brain dump about the problem, interested in thoughts and comments from others here about distributions they use/would consider in the server environment.