link

Useful thread on MSIE's table and div width bugs/quirks.

link

link

Interesting, short interview with Amazon.com founder Jeff Bezos. Mostly about innovation, and how Amazon prospers because of their focus on improving the customer experience. Innovation makes me feel warm and fuzzy inside.

Update: I think I figured this out. See bottom.

Quick semi-technical explanation: a character encoding is the way that letters and glyphs are represented in a computer. For example, you might decide that in order to convert a chunk of binary data into letters, you'd first split it into byte-long chunks. Since a byte has 256 possible values, you'd then map each of these to different glyphs, such as capital letter A or m dash.

I've been served pretty well by the belief that character encodings were things I'd never have to actually worry about myself; they'd always be so low-level that the operating system, or web server, or database server would just take care of it.

Well, not today.

It turns out that frassle pages that are filled with weird characters—like spaces and a Euro symbol where a dash should be—should show up correctly now, because I've configured my web server to serve them as UTF-8. The reasoning behind this, however, makes no sense.

Here is the experimental data. The first two rows are the old and new configuration on the server you're using right now. The only difference is that HTTP responses are now sent with a UTF-8 charset header. The last 4 rows are testing I've done on my own development server.

The columns from and to describe how the webserver is re-coding each page. Where these say none, I have tested disabling the recoding.

Machine	DB Coding	from	to	served	Result
production	ISO-8859-1	UTF-8?	ISO-8859-1	UTF-8	Good
"	"	"	"	ISO-8859-1	BAD
development	UTF-8 (Unicode)	UTF-8	ISO-8859-1	UTF-8	Good
"	"	"	"	ISO-8859-1	BAD
"	"	none	none	UTF-8	BAD
"	"	none	none	ISO-8859-1	BAD

Why, oh why does the page get converted to ISO-8859-1 and then get properly processed only as UTF-8? The conversion isn't just faking it; the length of the string actually changes as you'd expect in a conversion.

I'm guessing that something is wrong with the way I've been loading data into my DB. But what possible format could I actually be storing the data in such that treating it as UTF-8 and converting to ISO-8859-1 would produce correct UTF-8 encoded data?

Based on this experience, I suggest UTF-8 be renamed WTF-8. The new acronym has a structure that reminds experienced Unicode users of the original, yet includes a helpful hint to newbies that they are about to enter a world of pain and frustration.

---

Update later that evening: Joel on Software has an essay on this that has helped to clear my mind. Check out The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!).

---

Still later: I think I know why the conversion didn't work. The text I was testing with was picked up by my aggregator. Since I was in the "what you don't know can't hurt you" camp when I wrote the aggregator, it simply took the text from RSS (it was UTF-8) and dumped it into my database (pretending it's ISO-8859-1). Something like this must have happened:

1. Start with UTF-8 text.
2. Put it through a totally inappropriate filter, like ISO-8859-1 to UTF-8.
3. Now you have some incoherent garbage.
4. But if you run that garbage through a UTF-8 to ISO-8859-1 filter, you end up with your original text, in UTF-8.

This can explain the mystery of valid UTF-8 pouring out of a UTF-8 to ISO-8859-1 filter. I still need to find out where the first, invalid filter is; and I need to make the aggregator encoding-aware. But at least there might be a plausible explanation.

Jeez, I feel so dirty having written all that software without even thinking about character encodings.

I know I am tempting Murphy's law with a title like that, but if anything's wrong, I want to know. Well, unless it's the sort of thing that just goes away if you ignore it. Then I just want to ignore it.

So, I just applied a patch to a piece of frassle's infrastructure. I am hoping it has fixed a severe problem I and other users have experienced: losing the text of long posts.

Details: Intermittently, frassle would lose the text of posts when you tried to post. You'd hit post (or even preview) and frassle would save an empty message! This was the worst kind of bug, because it had terrible consequences and, because it was intermittent, I couldn't isolate it enough to test it properly.

I still can't test it properly, but I tested it as best possible and will be keeping my fingers crossed.

By the way, I can't believe I didn't do this sooner. I know some users have lost their work because of this, and am genuinely sorry for that.

link

Jessica has some interesting thoughts on using frassle within an intranet, which is something I've been thinking about a lot recently. Particularly important is her list of crucial features that frassle is missing. I'll let you know my plans for each below.

file/picture posting

• I think I could start this with a simple file attachment mechanism, and perhaps later expand it to e.g. be smarter about handling photos. Still, this is at least one version away.

user documentation

• I have started writing a lot of this, and over the next 2 months I think we will be in much better shape in this regard.

privacy options

• I think I have a simple but reasonable design for this and am going to try it tonight.

trackback

• I haven't quite figured out the design for this. Sending trackbacks is easy — frassle can just look at all your links and try to find trackback URLs at each one. But receiving trackbacks is tricky. Ideally, a received trackback would tell frassle where to get that blog's RSS feed, and then frassle would simlply start aggregating that feed and properly threading its notes with other notes in frassle. Trackback provides a lot of this information, but I have the feeling trying this will show me some unexpected difficulties and limitations. Therefore, I've been procrastinating until I can set aside a good chunk of time to experiment and back down if something goes dreadfully wrong.

link

That didn't show up as a reply because you changed with Link URL when you were entering the comment. Frassle uses that to thread discussions together, so if you change it, frassle assumes you are responding to something else.

link

A handy search engine that offers local information; seems to have a lot of useful detail in a consistent format. This is especially useful and unusual for restaurants, where the internet might have a lot of reviews, but they are scattered through lots of different sites; and not each site has the restaurants you care about.

link

Currently, you can't use a feed from a block in a noteset expression. For example, if you write the noteset expression:

feed="http://frassle.rura.org/rss?block=1"

which points at recent comments on my blog, your block will contain nothing. Nothing, Lebowski.

What you really need to do in order to get the contents of another block is copy over that other block's noteset expression. This is kind of hard to do unless you actually own that other block.

However, this trick you tried is the logical thing to attempt. So in the future, I'll have the publisher recognize what you're doing and replace that feed="…" expression with the original block's expression when feed you're referring to comes from a frassle block.

Here's a sloppy workaround, though:

1. Open, in your browser, the RSS feed for the block you want to duplicate. Example.
2. Near the top of the RSS file, you'll see an XML element called <description> which contains a description of the feed you're looking at. It looks something like this:

   <description>This feed is generated dynamically by frassle. See http://frassle.rura.org for more information. The noteset expression used is: responds_to( feed = "http://frassle.rura.org/Directory/rss?id=1" ) & not ( feed = "http://frassle.rura.org/Directory/rss?id=1" )</description>

3. Copy the underlined part, which is a noteset expression, into the noteset expr box in the block you're creating.

Note that if the block's owner changes its definition, your block will not change since you've copied the original search expression.

One thing I like about building software is that failure is normal. You have to try so many different approaches, and so many of them don't work. Some approaches fail at the idea stage; a few fail after you've spent days or weeks seemingly close to making them work. Each failure is painful.

If you were building a bridge or a putting out a fire, failure would be a total loss: materials wasted, property destroyed. But in making software, failure is actually a form of progress. When you're exploring the boundaries of what's possible, you're often going to set your sights beyond those boundaries. In making software, you're constantly reminded of this. As a result, you either get disparaged and start doing less risky, less creative work—or you learn to treat failure as an experimental result that confirms the value of the endeavor.

People who really care about making software know this. It's spread throughout the culture: good programmers, managers, execs, and even venture capitalists know the value of failure. For all the shortcomings of techie culture, it is this courage that redeems us. And because technology is an ever-increasing part of the world economy, the world is on a cultural path toward valuing failure and away from denigrating it. The effects ripple far beyond technology, making now a better time than ever to be creative.