Something that’s been kicking around in my head for a while, is the fact that I find most large web archives and blogs frustrating to various degrees. Including my own site!

The frustration has to do with what I would call, the “opacity” of sites to browsing, to discovery of information you do not know is there, and could not find because you aren’t searching for it directly. There are times I am in search of “information inspiration”- the serendipitous discovery of what I really wanted to know but didn’t know it.

Of course many sites have great search engines, and categorize articles by topic. But I feel that doesn’t go far enough. There are times when I am not even aware of the right WORD to describe a concept, but could recognize articles associated with it if I scanned them for a second or two. Or I am aware of the right word, but want some way to find ideas related to it serendipitously. This is what I call information inspiration.

Slashdot (and many other news sites and blogs) are date oriented. What happens to the articles after their moment in the sun, though is that they become increasingly buried. If you browse by topic, you still are unlikely to see them, since there are so many under each topic. Slashdot’s design discourages searching for topics.

First I have to say: don’t get me wrong. This is not meant as a critique of Slashdot per se: I LOVE Slashdot. I read it every day, and have for years. However, I feel like something is missing in general from our arsenal of navigation techniques on the web, so I am using Slashdot as an example here.

Go to Slashdot. If you are logged in, log out so you are anonymous, and customizations you’ve made will disappear. Pretend you are a new person, vaguely interested in – you don’t know what. But you want to see what there is here. So, you read the first page, then you may want to browse. You can easily see articles in no particular order dating back a couple of days by hitting the “older articles” area.

Back to the front page. On the left, you see a few sections listed:

sections.jpg

In this context I have no idea what good the “7 more,” “3 more” indications do me, since they are not clickable to reveal the missing sections.
In any case only a fraction of the topics are visible here – as well as the ones that are represented by the day’s stories which are displayed as picture icons at the top of the page:

topicIcons.jpg

The only way to figure out what those are, is to find the box labeled “Stories” (which is what I kind of thought EVERYTHING on Slashdot was anyway, making that title somewhat redundant, except perhaps to the programmer that set it up)

Listed there are:
slashdotMainMenu.jpg

Clicking the word “Topics” once you find it, will bring you to a page of topic headings. Clicking any one of those brings you to another date-ordered list of thousands of articles under that topic. There is a search box at the top, which allows you to search the list, which I find too little too late, since you can not search by topic from the front page, and it can take you quite a while to find that “topics” link even if you know it is there.

Both Slashdot and Amazon make it fairly difficult to browse by topic.

Once under a product type heading (i.e. “Books”), Amazon does try to make educated guesses about what you would like to see. As soon as you make any choices at all, you start getting items related to those choices. If you are logged in, it already knows enough to guess.

However, there is still a big problem with getting to buried information. If you do a keyword search for a book, and get a big list, there is no easy way to sift through the thousands of results.

I’ve often wished I could simply export the results to a database of my own and perform my own queries on them. All you can do is organize by date published, featured or not, A-Z (useless in a query that returns thousands), but there is NO way to jump to the middle. It seems amazing to me that they only provide a “More Results” button at the bottom of each page results, rather than a full featured browser of some sort.

Yes, there is an advanced search and power search, but they are not much better than the main search in some ways. For example, the other day I wanted to look up books by or on Velikovsky, including current ones that might mention him. Except, since I read his stuff years ago, I was absolutely unable to remember his name. So, I tried searching with combinations of terms, in both search and power search using:

“extinctions, biblical floods, Mars, Venus, climate change, catastrophe theories” and “publish date: before 1980” ( since I was just trying to find the author’s name at first, i wanted to exclude Alvarez-related books, and I knew that came in around 1980).

All searches resulted in either no results or thousands of results. I finally went back to Google, where I found it in minutes, using

mass extinction Mars theory

which pulled up a lot of very relevant results, including one in the first page with the word “catastrophism” in the title. This jogged my memory further, and I searched under that keyword.

About midway down the first page of results from “catastrophism” was Society for Interdisciplinary Studies which features resources on the subject of catastrophism. Right there on the page was a topic heading “Catastrophism and Velikovsky”. Bingo! I was able to go back to Amazon and find what I needed.

One might be able to argue “well, that’s just the way research is – you use whatever tools (Google instead of Amazon’s search) work to find what you need” but I think it is a fundamental problem with the way Amazon and many other sites are set up.

If I had been in a book store or library, I might have been able to find the book by scanning a shelf on related subjects, or looking at the bibliography of related books. Not quite so easy to do on Amazon, although you have many more books to choose from.

Amazon’s “people who bought this item also bought” feature, and their “browse subjects” features are both steps in the right direction.

To be continued…