The difficulties of operating an Olive Garden on the moon

(This is satire, fan mail, and a get-well card for Maureen Johnson. Here is some context for this nonsense.)

Maureen Johnson,

It is my displeasure to report that--at least temporarily--I've been forced to close my Olive Garden franchise on the moon. As you've always been such a strong supporter, I felt I owed you an explanation.

As the owner of many fine dining establishments in a variety of locales, I've found that running a restaurant has many challenges under the very best of circumstances. Running a restaurant on the moon has a few extra difficulties, and it goes far beyond the gravity problem (although breadsticks are notorious for floating right off the table).

There's only one health inspector on the moon (which caused plenty of scheduling problems), and he is MONSTROUSLY corrupt. When I refused to pay his "protection" fee, I found myself featured prominently in the "health inspection failures" report of the local newspaper. When I asked a fellow lunar restaurant owner how she dealt with the inspector, she (the restauranteur) said that she would just pay him. She regarded it simply as a business expense. I won't reveal which restaurant, but let's just say that I often imagined the inspector leaving that establishment, gleefully counting his money on his way out the door, saying "I'm loving it."

(This particular problem--the health inspection racket--seems quite specific to the moon. For example, even when the health inspector [DIFFERENT inspector] found at my Red Lobster restaurant in Atlantis that the kitchen, bathrooms, maitre D' station, bar, and dining area were LITERALLY CRAWLING with shrimp that weren't on the menu, he was entirely professional about the whole thing.)

(I had to close the Atlantean Red Lobster as well, but that's a different story. In retrospect, a seafood restaurant in Atlantis may have been a poor choice. Those folks prefer barbecue.)

Rampant smash-and-grab theft caused a problem with parking at the lunar Olive Garden. I've seen this problem at my Chili's restaurant on comet 67P/Churyumov-Gerasimenko, but it's far worse on the moon, where that nice little Philae lander would likely have been relieved of its harpoons before the driver even got his or her drink order. (There's a nearby pawn shop whose owner doesn't ask a lot of questions.)

Getting a liquor license on the moon is tough. All applications have to be approved by a council whose members are primarily other restaurant owners who don't want any new competition. Also, in a place where one day is nearly thirty times longer than on Earth, "Happy Hour" can result in some awfully thin margins. I thought serving cold beer at an Applebee's on Antarctica's Ross Ice Shelf would be difficult, but that's a cake walk (turns out that penguins are quite fond of Guinness).

And just try getting anything other than blue cheese on the moon. I mean I love blue cheese, but some of my customers prefer Parmesan on their salads. For someone who managed to find inexpensive chopsticks for a P.F. Chang's in freaking Asgard, you'd think cheese wouldn't be that big a deal, but it's a supply problem I still haven't solved.

So I'm hoping that this is a temporary setback. I've got a lead on a fromager somewhere in Mare Frigoris, I'm investing in some security cameras for the parking lot, and I'm sure the other problems will sort themselves out one way or another. I hope to reopen soon and that you'll honor us with your patronage. Thanks for all your support.

Management, Olive Garden on the moon

CentOS 7, openssh/openssl

Yesterday I finally gave CentOS 7 a try as a Virtualbox VM. (In the following, when I talk about a guest or a host, it's in the virtualization vernacular.)

I did what I usually do with VB guests: I gave it two network interfaces. The first is configured as NAT, so that the guest can reach the internet without the host needing a second IP for a bridged interface (bridging would be fine at home, but might cause me some trouble at work). The second is configured as host-only with a static IP, so that the host (and other guests) can initiate to the guest. (There’s probably a much easier way of doing this, but it’s worked so far.)

My CentOS experience is primarily with CentOS 5, and several things were really different in C7. (CentOS and Red Hat documentation is typically pretty good and will no doubt help me through some of the following. These are just some of the things I’m stumbling on at the moment.)

There’s no /etc/cron.daily/rpm, which creates a list of packages in /var/log/rpmpkgs. I use that a lot, so I copied that over from a C5 box.

I had a pretty hard time with networking. Neither interface seemed to come up on its own at first. I had to set ONBOOT=yes in the corresponding /etc/sysconfig/network-scripts files, and then the second interface mangled the first interface’s NAT connection. I ended up setting ONBOOT to yes for the first interface (the NAT connection) and to no for the second (host-only) interface. I put an ifconfig statement in rc.local to bring up the second interface, and that (eventually) worked.

ifconfig, netstat, and probably a bunch of other useful stuff is in the net-tools package, which isn’t included in a minimal install. And although there’s an rc.local, it’s not executable, and won’t run at boot until you “chmod +x” the thing.

And the interface names are now really weird. Instead of something memorable, traditional, predictable, and sensible like eth0 and eth1, now they are called enp0s3 and enp0s8. (I just had to look those up, because I couldn’t remember them.)

The new C7 guest has a very long list of iptables rules, but /etc/sysconfig/iptables doesn’t exist, so I don’t know where those rules are coming from. Thankfully port 22 is open by default, but I don’t like to run openssh on the default port, so at some point I’ll need to figure out how to fiddle with iptables rules.

I use GNU screen all the time. (I know the cool kids like tmux, but, frankly, screw them.) I typically have a screen session in which I’m logged in to several different hosts, and each window is named for the remote host. C7 rewrites the window name in screen, so “Ctrl-A ‘, hostname” no longer works. I don’t know if I need to (somehow) tell screen (on my Ubuntu host) not to allow the window process to rewrite the window title, or if I need to (somehow) tell bash in the C7 guest to be less assertive.

I’m also having some trouble building openssh from source in C7. The version of openssh that comes with C5 lacks some desirable features in the newer versions, so we tend to build it from source. In just the last version or two of openssh, something has changed such that it won’t build against the version of openssl that comes with C5. So the other day (before messing with C7) I built the newest version of openssl on a C5 box and built openssh against that. That worked, but I see now that by default openssl doesn’t create shared libraries, so the openssh I built linked to openssl statically (which made sshd nearly three times bigger than a dynamically-linked sshd).

So far I’ve been unable to build openssh against a source-built openssl on C7. I get one error if I try to link statically, and another error if I try to link dynamically. The version of openssl that comes with C7 is pretty current, so I could just build against that and probably have no problem. Likewise I could just use C7’s version of openssh. But although I’ve enjoyed the stability of C5, everything about it is pretty old at this point. I think that in the future I’d like to build many or all network services from source. Since openssh, apache, stunnel, and several others need openssl, I’d like to keep that current, too.

So I have some work ahead of me. I think C5 hits end-of-life some time in 2017, so I’ve got some time, but the C5 EOL will probably sneak up on me if I let it.

PHPUnit via phar with suhosin

The PHPUnit developer has announced the end of life for the PEAR installation method for PHPUnit. So I tried the phar method, but phpunit (and phar) would fail silently. After a little searching, I found a useful post* suggesting that suhosin might be the problem (I only have PHPUnit on my development platform, but I also have suhosin there, so that it more-or-less matches production). That post's suggestion of adding suhosin.executor.include.whitelist=phar to php.ini did the trick.

*That post is for an entirely different package, but it was the same problem I was having.

Problem with twitter's login verification feature (resolved)

This is another "maybe this will help someone" kind of post. Long story short--if you stop getting login verification text messages from twitter, try texting GO to 40404.

At some point I enabled the login verification feature for my twitter account (@carl_welch). So whenever I want to log in to twitter.com on a PC or laptop, I type my username and password, and then twitter sends a 6-digit numeric code to my cell phone via SMS. Then I type that code into the form on twitter.com, and I'm logged in. I have my browsers set to delete cookies when I close the browser, so I end up doing this somewhat frequently.

(It works a little differently logging in to the twitter app on my phone. I have to log in to twitter.com on a laptop or PC and generate a one-time password that I use for logging in on the app. I generally only have to do this once, because I don't log out of the app on my phone.)

I got a new phone a couple of weeks ago: same number but a different provider. And I immediately stopped getting the login verification messages from twitter. So I couldn't log in. I'd managed to get logged in on the phone's app before restarting the browsers where I was logged in to twitter, so I could still use twitter while this was going on. But before I realized that I had a problem, I'd restarted my  browsers, and I couldn't log in (on a PC or laptop).

I submitted a help request to twitter explaining the problem, speculating that it might be because I changed provider, and asking them to disable login verifications for my account. I'm quite disappointed that--other than automated responses--I never heard back from them at all. Lame.

So I kept poking around on their help pages, and I stumbled upon a page talking about how to tweet via SMS. Seems like you start that process by adding a phone number to your account (which I'd already done) and then texting GO to 40404. For lack of a better idea I tried doing that on my new phone. And login verifications started working again immediately.

Lucky.

Software Choices

I switched to Xubuntu a few days ago, and I've been trying a few changes in some of the other software that I use, too.

Audio Player

After using rhythmbox for a couple of years, I really tried to like gmusicbrowser, but I just couldn't. Even with the rhythmbox skin, I just didn't like the way the browser worked. I like to click on an artist and see the albums, and then click on an album and see the songs. gmusicbrowser didn't seem to do that (or I couldn't make it do that, anyway).

So I looked at the Best Audio Player page of the recent readers' choice awards from Linux Journal to see what other people are using.

I tried VLC, but it seems like more of a file player. I didn't see a way to import and browse my music library. I may not have given it enough of a chance, but I was disappointed with it.

I rejected Amarok, because I'm not running KDE. I looked at installing banshee, but I was turned off by the long list of mono-related dependencies.

So I gave up and went back to rhythmbox. To my pleasant surprise, the list of dependencies didn't look too long (no longer than banshee's, anyway), so I decided to stick with what I know.

CD Ripper

I got a couple of CDs as gifts last week, and I wanted to rip and encode them, so I tried ripperX (which I'd used previously). For ripperX to work correctly, you need cdparanoia (to rip the CD to WAV files) and lame (to encode the WAV files to MP3 files). I'd installed cdparanoia, but had overlooked lame. When I tried ripping a CD with ripperX, it dutifully created MP3 files without a word of complaint. But then the audio player refused to import them. It took me a while to figure out that lame was missing, and it's disappointing that ripperX couldn't give me an error message about it. Even after installing lame, ripperX wasn't adding the ID3 tags: the CDDB lookup worked, but the generated MP3 files had no ID3 tags. That's important, because audio players use the ID3 tags to organize the music files.

Next I tried asunder (which I'd also used previously), and it worked on the first try, ID3 tags and all.

BTW, easytag is good at fixing problems with ID3 tags (and so is id3).

Personal Financial Manager

I've used grisbi for years, but I've never much liked it. I looked at GnuCash a year ago. I even exported a QIF file from grisbi and imported into GnuCash. But despite the excellent documentation from GnuCash, I was overwhelmed by the transition, so I stuck with grisbi for another year.

A couple of the gripes I had with grisbi (I was running v0.5.9 in Ubuntu 10.04) were minor, but annoying. The transaction dates are formatted for Europeans, and I just couldn't get used to that (in fairness, it looks like that has changed in the meantime). And I never had much success with the reporting feature. I typically ended up exporting the data in CSV format and then getting what I needed from a spreadsheet.

A more significant problem involved what is for me a common practice. When I go grocery shopping, I typically buy groceries and beer or wine and get cash back at the register. The credit union sees that all as one transaction, but I like to track groceries, booze, and cash separately. In grisbi I'd have to make three separate transactions and then remember to put them together when reconciling with the bank statement.

So this year I started a little earlier and read through the sections of the GnuCash documentation in time to make the transition at the new year. GnuCash has "split transactions" which should address my gripe about what to do with my grocery bill. I'll still have to do arithmetic to figure out how much is booze and how much is other stuff, but it may make it easier to reconcile when the end-of-month statement arrives. Dates look the way they "ought to," and the documentation makes the reports look easy and helpful. This one is a pretty big experiment, so we'll see how it goes.

Other Stuff

I think I'll try LibreOffice this time, rather than OpenOffice. I don't use either enough to feel strongly one way or the other, but it seems like I've read that LibreOffice may be getting more active development. *shrug*

In the past I've used XEphem for my virtual stargazing, but I've recently discovered Stellarium. I like Stellarium, and you don't have to build it, so I'll give it a try.

Xubuntu

I've been running Ubuntu 10.04 for a couple of years, but it goes end-of-life soon. I knew that upgrading would more-or-less mean learning a new window manager: Unity or something else. I decided to try going with XFCE in Xubuntu 12.04 (which will have support until April 2017).

I have so far been pretty happy with it. I backed up everything I could think of to a second hard drive, and then did a fresh install off of a USB drive. It didn't take long, and then I spent a couple of hours restoring files and setting up software and stuff, such as...

• installing google chrome and importing bookmarks (I'd exported them prior to the upgrade)
• restoring ~/.purple so that pidgin would work
• installing postfix and configuring it to relay off of gmail, so that logwatch and my cron jobs would land in my gmail inbox (I'd already done this, so I just had to restore a couple of files)
• installing VirtualBox and restoring ~/.virtualbox (and my guest machines booted without much complaint)
• pointing gmusicbrowser at my MP3 collection and skinning it to look like rhythmbox (there are some minor differences I'm still getting used to, but it looks like the playlists I exported from rhythmbox import right into gmusicbrowser, so that's a pleasant surprise)
• setting up an encrypted subdirectory in $HOME

I find that I don't much care for the panel at the bottom (the one that's supposed to look like OS X), so I may end up removing it. And I'm finding that the keyboard shortcut for "maximize window" is flaky for some reason. But those are about the only two wrinkles I've found. It was otherwise pretty painless, and it took less time than I expected.

Red Hat and CentOS process locks do not belong in /var/lock/subsys

I have a bash script that runs via cron every night. It's a software package repository mirroring process, so it has the potential to run for a long time. I don't want two of them running at the same time, and I want to know if one is running for more than 24 hours.

So the first thing the script does is to check for the presence of a lock file in a particular location. If the lock file exists, that means that the previous instance didn't complete, so it spits out an error message (which will land in my email) and quits. If the script does not find the lock file (which is the normal condition), it creates the lock file (via touch) on the very next line. The very last thing the script does is to remove the lock file.

if [ -f /var/lock/subsys/mirror ] ; then
    echo 'previous instance running--quitting'
    exit
fi
touch /var/lock/subsys/mirror

# long-running mirroring process...

rm -f /var/lock/subsys/mirror

We have scheduled downtime from time to time to do operating system upgrades and other maintenance. I typically create the lock file manually at the beginning of maintenance, because I don't want my local repository changing while I'm running updates. And then I manually remove the lock file at the end of maintenance.

We did maintenance last night, and the server that runs this mirroring process was having some kind of problem. We ended up rebooting the server (which, in this case, resolved the problem we were experiencing).

Turns out that part of the reboot process on Red Hat (and CentOS) is to clear out the /var/lock/subsys directory, which is where I'd been putting the lock file for the mirroring process. Oops.

Fortunately, the reboot happened after the cron job, and so the cron job didn't run unexpectedly. But that was just a matter of lucky timing.

So today I'm moving the lock file to /var/run. And I may as well make it work like the other files in that directory. So instead of

touch /var/lock/subsys/mirror

I'll do

echo $$ > /var/run/mirror.pid