I expect a more critical opinion from Ben

Comment Spamalot:

But it’s not really anyone’s fault but the spammers. Yes, Six Apart could improve the load the scripts place on the servers, (and perhaps they are. Who knows? For a blogging company, their own blogs are very quiet. Sensibly, in my opinion, but that’s another story.) The targeting of MT, however, shows nothing more than its popularity. It’s certainly not a security issue on a par with Internet Explorer: the comment spam is only using a facility to the purpose to which it was built.

Eh? I’m not sure I am willing to concede this point: my own struggles with MovableType are still fresh in my mind.

The blithe statement that “[t]he targeting of MT, however, shows nothing more than its popularity” is disingenuous: why, after two years of this, has so little (anything?) been done about it? Why can’t admins reliably take evasive action?

There are two issues here for site hosting MT, both manifesting themselves in disproportionate system resource usage.

  1. MT is slow and gets slower as its content database grows. North of a couple thousand entries, things bog down, the rebuilds take ages, comments get repeated as users run out of patience with sites, browsers time out (try Safari and it’s hardcoded 30 second timeout on a big site).
  2. The system has not been designed or tested so much as built. Some of the simpler (and even recommended solutions) don’t work as documented (read: at all) and it makes me wonder what testing has been done with each new release.

The solutions to these are a. throw more resources at it, or b. tune the system with, say, mod_perl. Option a is not a realistic choice for hobbyists or hosting services who are unprepared for weblogs to become a regular item on the staff meeting agenda. Option b is sensible, on its face, until you realize that (for 2.x) some of the popular extensions/plugins (MT-Blacklist, for example) don’t work in a mod_perl environment. An even easier option — simply renaming the comment and trackback scripts — won’t work either in a mod_perl environment: there are some hardcoded references to those script names somewhere in the bowels of MT’s .pm files that preclude this working either.

I think Ben needs to re-examine this: does he really think MT’s problems are so trivial or at best, an artifact of their success?

not all hardware is flaky

I needed to add a DVD burner to my circa 1997 PowerMac G3 (the funky Blue & White) and after tracking down the disassembly instructions, I was successful. The internals of the machine both look and feel more solid and, well, designed, compared to commodity PCs. And the drive was recognized (though System Profiler claimed it wasn’t a burner, while Toast proved otherwise) right off. Now to find some more disk for it (I hate to dump money into such an old and obsolescent box, but the $1800 I would need for a G5 iMac isn’t forthcoming anytime soon: say, could you click on an ad on your way out? Every little helps . . . though Google’s T&C forbid my saying as much).

This old box may be around for years yet (I still have a 1995 vintage 9500, upgraded to a raging 233 MHz 604 processor). These boxes lack the sheer design appeal of a Cube, but they still run and reliably enough to keep them around. I have retired one Intel box that is newer than either of these Macs and the current host that runs this website may see the same fate before too long.

More on top 20 IT mistakes and software engineering

More on top 20 IT mistakes and software engineering:

More on top 20 IT mistakes and software engineering

Malcolm Davis over at the java.net weblog community commented on the Top 20 IT mistakes feature I wrote a few weeks ago and picked the following “favorites”: mismanaging software development, developing web apps for IE only, and clinging to prior solutions.

I didn’t see this until I got home tonight, but I encountered an example of one of these today. I am helping with my school yearbook (elementary schools have yearbooks now?!) and we are going to do it all digital this year. But the web application requires IE. It won’t let you use anything else. I’m underwhelmed. I plan to grouse to the contact at the yearbook company about it and forward on some stuff about how organizations are rejecting IE or barring its use altogether.

Crypto-Gram: May 15, 2001:

If at all possible, don’t use Microsoft Internet Explorer.

[ . . . ]

If possible, don’t use Microsoft Windows.

flaky PC hardware

I think my old (almost 5 years!) PC server is nearing the end of it’s useful life.

I am seeing lots of signal 11 errors (generally hardware-related, specifically memory) under load, and for some reason, my parallel port (stop snickering) disappeared. Printing wouldn’t work and reinstalling the printer through CUPS was a non-starter without the parallel port. For some reason, it went into such a deep state of rest that it couldn’t be roused: the only way to restore it was to make sure the printer was on and awake (it’s an old HP1100 that doesn’t have an on/off switch) and then reboot.

Once everyone had been re-introduced, everything was fine.

But I’m still thinking things are not going to get any better. Might be time to look into something more robust and take advantage of Moore’s Law at the same time. Looking at PriceWatch.com, I can get a whole lot of system for $300-400.

At the same time, I have been giving some thought to the home-brew PVR/MythTV: I’m not sure I have a use for one but I suspect if I had one, I might find uses for it. I suspect combining the two of these is not a great idea . . .

search/replace with SQL

I had occasion to root out and replace all instances of my old identity (paulbeard.no-ip.org, where I found some cheesy placeholder page: thanks, no-ip.org) with my current domain (relative links would have made this unnecessary, but then I would never have learned this, would I?)

update crank_posts set post_content =( REPLACE (post_content, ‘paulbeard.no-ip.org’, ‘www.paulbeard.org’));

One quick Googling and hey, presto. T’would be nice to swoop through and pass all my old entries through some kind of html-sanitizer/tidier, but I’ll deal with that later.

will work for music . . .

PayPal – Use PayPal on iTunes—Get Five Free Songs!:

With a catalog of over one million tracks, iTunes lets you buy, download, share, and enjoy the music you love, both online and on the go. And PayPal makes it easy and secure for you to make and receive payments online.

Now, when you sign up for a new, free iTunes account and pay with PayPal in the iTunes Music Store, you’ll make your online music experience better than ever. And if you’re one of the first half million members to do it before March 31st, you’ll get five free songs.*

But I already have a Paypal account and I’ve spend a coupla hundred bucks at the iTunes Music Store . . . . gah.

I set up payments through PayPal, rather than a credit card: makes sense since I get paid through my PayPal account. But it would have been nice to get comped the 5 free tracks.

fun with rrdtool

I discovered (belatedly) that my weather plotting graphs hadn’t updated in forever — a problem with how the data was expected: something changed in rrdtool and I never noticed when it stopped working — and while I was fixing them, I decided to take a stab at moving the hits/errors graphing from mrtg to rrdtool. Hitrrd
So I edited my small script from a few days ago. (It’s not all the useful with just two datapoints since that’s all mrtg provides: I haven’t decided yet what to add as additional data for this but I do have some ideas for different system graphs. Watch this space.)

Then I created an rrd data file:

rrdtool create httpd.rrd –step 300
DS:hits:GAUGE:600:0:100000 RRA:AVERAGE:0.5:1:1:1200 RRA:MAX:0.5:12:2400: RRA:MIN:0.5:12:2400

DS:errs:GAUGE:600:0:100000 RRA:AVERAGE:0.5:1:1:1200 RRA:MAX:0.5:12:2400: RRA:MIN:0.5:12:2400



RTFM for more information on how this works.

Then the data collection and updating needs to be managed:

#!/bin/sh

cd /usr/local/etc/rrdtool/

/usr/home/paul/bin/hitrrd.pl > httpd.out

/usr/local/bin/rrdtool update httpd.rrd `cat httpd.out`

rrdtool graph httpd.png –title “httpd hits and errors per minute ” -s -3days

DEF:hits=httpd.rrd:hits:AVERAGE DEF:errors=httpd.rrd:errs:AVERAGE

AREA:hits#00FF00:”hits” LINE2:errors#FF0000:”errors”

And that’s it: if you like, you can make multiple graphs for different periods (like mrtg does by default) by repeating the last statement and adjusting the output file and the interval to be graphed.

The results are here. And the full script that generates four graphs is below. I leave migrating data from mrtg to rrdtool with the attendant alterations to the internals of an rrd file as an exercise for the reader but with these hints: log2rrd and rrdtool tune.

Continue reading “fun with rrdtool”

bad citizens

Spammers are playing havoc with my site and probably others today:



[/usr/home/paul]# wc -l /opt/weblogs/httpd-access.log

29019 /opt/weblogs/httpd-access.log

(root)-(01:52 PM / Thu Dec 09)

[/usr/home/paul]# grep -c ” 404 ” /opt/weblogs/httpd-access.log

26017

 Mrtg Red Red-Httpd-Day

Everyone of these spam requests comes in as “Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)” and I have no reason to doubt it: anyone know of any magic packet that push these boxes over? And why don’t their network providers realize this is going on? The rates of fire are far too high to be legitimate web requests. Time to look at mod_throttle.

no excuses

Reading Neil Gaiman’s journal in my aggregator is acting as a gentle but persistent goad: he provides these little glimpses into how he works and forces me (and probably others) to face up my own laziness or lack of focus.

Radio Silence:

Had an excellent writing day today until derailed by email, and hope to have an excellent writing night, or at least a good one. Right now, at least, the book is behaving. I’m still not sure why or how Fat Charlie is going to get out to the Caribbean, mind, but I’m fairly sure it’ll take care of itself when I get there, and it probably has something to do with Mrs Dunwiddy anyway.

Writing is work, just as gardening or cooking or programming: it’s sometimes no more clear how any of those tasks are going to work out, either, but they sort themselves out as they go. Perhaps it’s time to follow a NaNoWriMo program year-round . . .