Liver function tests
Yesterday my doctor told me that I have a lifelong, incurable disease, and I was overjoyed by the result. It was the best possible news.

It all started about six weeks ago, when I had a blood-test taken by the doctor. I was feeling great, and I'm sure most people in my situation would have been out doing something enjoyable, rather than being prodded with a needle in a doctor's surgery. However, I believe in catching problems well before they start. A little blood-work now and again is a great way to spot issues before they turn into more serious problems.

In any case, I hadn't expected my blood tests to show anything abnormal, so I was rather surprised when my results came back with elevated liver enzyme levels (ALT, ALP and GGT). This in itself was no great cause for alarm; I was showing no other signs or symptoms of ill-health, and there are many acute reasons why they could be elevated.

A discussion with the doctor convinced me to give up alcohol for the next month, just to be on the safe side, at which point a second blood-test would be taken. So I went around my business pretty much as normal, although with a few regrets that I couldn't enjoy the occasional beer after work.

My second blood test results were more interesting, and it was ironic that due to an IT problem the surgery only had a partial copy of the results. However from that partial transcript it was clear that my enzyme levels were still elevated, and that I didn't have any of the nastier forms of infectious hepatitis. However we were at a loss as to why my levels were still high.

The doctor scheduled me for a full abdominal ultrasound. The ultrasound machine was one of the coolest toys I had seen in a long time. A small amount of gel, a hand-held sensor, and one can take incredibly detailed images of inside a patient. I spent half the examination gawking at the machine and asking how it worked, as well as discussing how the organs are imaged and measured.

The examining doctor did a very thorough job, with images taken of my spleen, kidneys, gall-bladder, aorta, and of course my liver. Lots and lots of liver images. The result was that every single organ looked completely normal in size, shape, and texture. No signs of disease, no signs of inflammation.

Back to the doctor with my ultrasound results, and what I was expecting to be a perplexing situation for the both of us. However this time my doctor had the full results in front of her, and discovered that I tested positive for Cytomegalovirus, or CMV.

A person living in the USA has between a 50 and 85 percent change of contracting CMV by the age of 40, and I imagine the conditions in Australia are much the same. The virus remains dormant for most of its life, but will occasionally flare up causing flu-like symptoms, particularly when the patient's immune system is weakened. When it does so, it also causes mild, reversible hepatitis. It has no long-term health consequences, and is not considered a serious disease.

Now everything makes sense. I had recently been showing flu-like symptoms near the time of my blood tests, and the elevated enzyme levels without apparent inflammation or damage to the liver is a classic sign of CMV. I'm getting some more bloodwork done to make sure, and should have a conclusive result by next week.

What's interesting about this whole affair is that I've known that I've had CMV for years. I'm a regular blood donor, and CMV is one of the routine tests performed on blood donors. When I showed up positive many years ago I had queried the blood bank about it, and they simply described it as "a type of flu, not something to be concerned about".

If my doctor and the blood bank had actually compared notes, then my CMV status would have been already known, and a test for activated CMV could have been performed immediately, potentially saving me a rather uneasy period where I knew there was some "mystery disease" that was causing problems with my liver. Still, I'm very glad that it looks like I'll be given a clean bill of health.

Machine "Recovery"
Machines should be designed with consideration for when things go wrong. Now, let's pretend that you have an "appliance". It has an LCD screen, rather than a keyboard and monitor. If you're lucky, console on serial has been enabled. It has "no user servicable parts inside", and while it runs Linux underneath, it's primarily marketed at non-Linux gurus.

You're picking a filesystem to place on one of these "appliances". Do you choose a filesystem like ext2/ext3, which is used by practically every Linux system worldwide, has good stability, and excellent recovery tools? Or do you choose something else?

For these appliances, a different filesystem was used. This filesystem is a fast journaling filesystem, but it has some problems when things go wrong. It doesn't have a real fsck program, it just assumes that it's always good, all the time. So when there is a problem and corruption does occur for whatever reason, finding and repairing that can be difficult.

This brings me to my situation. There was corruption on the root filesystem. There was also a repair tool, on the root filesystem. The repair tool will not execute on a mounted filesystem, not even one mounted read-only. Normally this is fine, boot from external media and repair, but these appliances have no option to boot from external media. There's nowhere for that external media to go.

So, my options are to get a new copy of the filesystem tools (which can allegedly repair a read-only mounted filesystem), or to pull the disks (RAID-1) from the machine, get them mounted in another machine, get the tools on that, get them working properly as a RAID, repair the filesystem, and put them all back into the appliance that contains no user-servicable parts.

I decide to try the newer version of the tools.

Being a rather cautious person, I take an image of the root filesystem first, and ask the new tools what they would do with it. Well, they'd make some pretty big changes to the superblock that look very suspicious; then they'd tweak a few bitmaps here and there; and then they'd segfault.

Having your filesystem repair tool segfault on step two of seven in the repair process is not a way to fill your system administrator with confidence. It's very, very bad. Some people have nightmares where they're drowning, or speaking naked, or their house burns down. Sysadmins have nightmares that their filesystem tools segfault part-way through changing a filesystem.

Now the tools do say they could potentially segfault if run in no-write mode, since they can't repair the things which later parts of the tools expect to use, but my faith is still quite shaken. Okay, let's try the tools again on the image in write mode...

The tool walks through the image, makes a number of changes, and says all-done. I'm not convinced that relocating the entire superblock to a totally different location was a good idea, so I try to mount the file using a loopback device. No joy here, the kernel on these appliances can't handle a loopback image of more than 2Gb. Welcome to 32-bit limit land.

At this point I'm pretty much decided that running the repair on the real filesystem is a Bad Idea, but I decide to continue a few more tests on my image. I mean, it should contain a perfectly good filesystem now, right? So if I run the repair tools again, it should come up all clean? Wrong!

Running the repair tools again claims there are still errors (different ones, and much less) on the system. It also claims that they're fixed, but running it again reveals the same errors in the same places, right where I feel the superblock should be sitting.

Needless to say, I'm recommending that data and services me migrated off this appliance and the entire thing rebuilt/reinstalled from scratch.

I'm only glad that these machines looks like they're going to be retired before the end of the year, and replaced with new servers running ext2/3, with ports for a keyboard and monitor as well as a serial console, and the most blessed ability to boot into single-user mode.

Luxury!

I have time!

After what seems months of ending training, fire-fighting, software-development, and paperwork, I finally have a more free and flexible schedule. It's glorious! I slept in the other day, have been playing network games, writing conference papers, and otherwise having a wonderful time.

Sure, clients still call all the time, but with simple things that are easy to diagnose and fix. No big disasters, no crushing deadlines. It's just great.

Now, I have a big pile of work that hasn't been done because I'd been so busy. Some of that's quite important, and it looks like I'll be able to knock it over with plenty of time to spare. That's a really good thing, because it's means no painful deadline crunches later on.

I've even got confirmation that my replacement wetsuit is on its way, although I'm somewhat unconvinced about cold-water diving in winter.

Unsurprisingly, my body has reacted to all this wonderful time and relaxation by catching a very minor cold. I don't know why people get sick when they stop being stressed and busy, but it's certainly more convenient then the other way around.

Fancy a career change? Become a SysAdmin
Are you sick and tired of being recognised for your hard work? Do you hate working normal hours that allow you to see friends and loved ones? Would you like more stress in your day? Do you want a job where people only notice you when things go wrong, and consider you redundant when things go right? Do you want urgent and stressful phonecalls while you're on vacation? Do you want the satisfaction or never being given the tools or time to complete your work properly?

If you answered 'yes' to all of the above, then you may wish to consider a career as a system administrator. These brave young souls are ushering in a new era of reliability and stability for systems worldwide, or they would if their requests for monitoring and backup systems were ever taken seriously.

Yes, as a system administrator you have a lot to look forward to. Phone-calls at 3am, irate users who don't want to read the manual, and blame for the lack of backups, even though you've spent a year trying to get funding for them. Best of all, when everything goes well, people will start asking why they even need a system administrator in the first place.

So what are you waiting for? Leave that satisfying desk job behind, and join one of the elite ranks of system administrators! You have nothing to lose but your peace of mind.

Tips for businesses - scheduling with others
Let's pretend that you've come across a dynamic, active business that provides Perl training in Australia, and you're eager to market their courses to the rest of the Asia-Pacific region. You have an agreement on how to handle bookings and payments — or at least you've been sent a proposal and you haven't objected — so it all looks pretty good.

The training manager informs you that they're very busy in the coming year, so please make sure that she gets any proposed dates ASAP to make sure there are no conflicts with existing arrangements.

Do you:

1. Work with the training provider closely to arrange dates.
2. Don't liase with the training provider, but instead look at the training provider's freely available on-line calendar and organise courses that don't conflict.
3. Ignore the training provider and their calendar, organsie some dates that may or may not conflict, and send them notification after advertising has commenced.
4. Ignore the training provider and their calendar, and instead organise dates that suit you. Don't tell the training provider what the dates are, even though you've started to advertise the courses. When the training provider asks for dates ignore them for the first six requests, and then finally respond.

If you answered 'd', then congratulations! You're in the same league as many other international training organisations.