link

At the Berkman meeting on 12 Feb 2004, we continued our project to understand and design the ideal aggregator. The following is my summary of that discussion. I meant to post this several days ago and to make it strictly factual, but the passage of time has made my memory fuzzy, so I think it's more useful for me to abstract and organize than attempt to create minutes. I will also toss in some opinion, which I will try to make clearly discernible. I hope this will be useful and welcome your comments.

Current Options and Limitations in Aggregators

We considered the current state of the art in aggregators. There are two main styles of aggregator out there right now:

• three-pane aggregators, such as FeedDemon or NewsGator, which typically present a list of feeds on the left of the window, and split the right part of the window vertically, displaying the titles and dates of posts on top and the contents of one post in the bottom; and
• one-pane or blended-feeds aggregators, such as the one built into Manila, Feedster, and frassle, which blend incoming posts from a number of weblogs into a single stream.

One issue that was brought up is how aggregators handle scalability. In my opinion (and I actively use both kinds of aggregators) this is a serious challenge for either approach. A three-pane aggregator with too many feeds will tend to gather a number of feeds that simply don't get read. I tend to subscribe to all sorts of feeds that sound interesting, but I don't read them thoroughly right away, and after they fail to make my first-pass reading list for a week I forget what they're about, and lose any motivation I once had to read them.

In a blended-feeds aggregator, on the other hand, I am reluctant to subscribe to too many blogs. I need to keep the average size of the aggregator posts view short enough that I can efficiently skim it every few hours; if it has 150 new posts each time, I can't even whiz through it in the kind of time I'm willing to allot.

One possibility is that the 3-pane style offers a better hint toward a scalability solution because I can implicitly capture information about how important a certain feed is by how it's organized in the left pane. Another possibility is that the 3-pane aggregator is simply offering me an option of denial: I can subscribe to a feed without risk of inflicting too much new reading on myself, but as a result that subscription becomes meaningless. Because a blended-feeds aggregator forces me to view a bunch of different feeds as part of the same stream, I don't have that cop-out. I think there's a lot to learn from each approach.

Where it's Located

Another major dimension of aggregator design is where the software is located and accessed. The two main approaches here are desktop and web-based. A desktop aggregator tends to offer more control and a richer user interface. A web-based aggregator tends to offer more availability (since you can use your aggregator from any web browser anywhere in the world), but its UI is limited to typical web browser capabilities.

How are items selected?

Although we typically think of aggregators as gathering posts from a list of subscribed RSS feeds, there are actually a few different sets of criteria that can drive an aggregator:

• manual— the typical approach, where a user maintains a list of feeds himself and the aggregator gets all posts from each of the feeds.
• automatic— the aggregator software gathers feeds or posts that match certain criteria. For example, you might want all feeds written by authors in the Boston area, or all posts mentioning your name or your company's product, or posts to the top 100 feeds in Share Your OPML.
• curated— an individual maintains a list of feeds for others to use. For example, a medical librarian might gather RSS feeds relevant to different teams in a pharmaceutical research lab, and thereby offer her patrons an easy way to stay apprised of new developments in their fields.

It's interesting to note that some of these options seem more like feeds than aggregators: the simplest case is feedster, which will allow you to make an RSS feed of its RSS search results. Although the output is RSS, the RSS contains an aggregation of many different feeds. Another cool example is w4 k-collector, which organizes posts from a multitude of feeds by topic and offers results in RSS as well as in your browser.

Another area where I see some potential is in bridging between manual, automatic, and curated styles. Can we have an aggregator that delivers posts that are relevant to you (manual), but with little effort (automatic) and leveraging the expertise of others (curated)?

The Grand Challenge

What would a great aggregator look like?

We don't know yet. Write down ideas, draw pictures, and share them with us. You can

• come to the Berkman meeting tonight or any Thursday,
• post to the AggregatorWishList Wiki Page on Rick's server, or
• email me and I will bring up your idea at the meeting.

Got home 20 minutes ago from the Thursday meeting. I'm exhausted.

Josh and I left work at 3pm, drove to Harvard to hear a talk by Lawrence Lessig, and chatted with Mike Walsh during the reception. Afterwards the three of us went to Berkman into a meeting I rather enjoyed. The dinner afterwards (at Cambridge Common) was pretty good too.

I'll write more on the Lessig talk soon. Meeting notes will show up tomorrow, including notes I took on the project to understand and design the ideal aggregator. As usual, I came away from the meeting with a couple of really cool ideas for future features in frassle (say that five times fast).

Wendy and Joey are the cutest couple. Awww. Joey played an accordion/baritone rendition of "Hit Me Baby One More Time" at dinner, which I recorded on my new digital camera. The video (with sound!) came out perfectly, and soon the whole world will share in my first experience of The Accordion Guy!