In the beginning there was Yahoo. Yahoo categorized web sites into logical groups and categories and it was good.
The categories allowed people to navigate and find information using a logical structure but it wasn’t long before this approach started to run into problems. The first problem was that certain web-pages and even certain categories didn’t seem like they fit into a particular location in the hierarchy. Many pages seem like they could fit into all sorts of locations. In addition different editors may make different decisions on where something belongs. It wasn’t long before the directory was getting confusing to navigate and as the directory grew people relied more and more on search.
Altavista, Excite, Lycos, MSN, Yahoo, and more recently Google poured themselves into search technology to solve the problem of categorization. By indexing and ranking pages users could perform a search and find any page that has something to do with a particular word. This was a great automatic approach to categorizing and is still the most popular way to find stuff. But searching has a serious problem. Search engines can’t tell what something is about.
For example, let’s say I write a joke, it’s particularly funny, witty and clever. People reading it may think it’s funny but anyone using a search engine to find things that are ‘funny’ would be unlikely to find my joke since it didn’t use the ‘funny’ or ‘witty’ keywords. This is where Folksonomies can help.
The word Folksonomy is a combination of Folklore and Taxonomy. The idea is that you allow the community to categorize the page and add keywords or tags to your page. The idea is similar to the original Yahoo categorization only anyone can be the editor.
Folksonomies need a fairly large population to properly work but they allow the actual users of the data to organize it dynamically by adding text ‘tags.’ If some people think of something as a “Funny” and others think of it as “Penguin” and still others think of it as “Lame” the democracy of the tags allow the page to be properly categorized. The more people add tags the more trust that the popular tags are correct. As tags get added you also learn more about the content of the page. You also can easily find related pages that have similar tags. In addition it’s harder to spam or fool users with fake pages. Traditional search engines have a hard time detecting a bogus websites but real users can see a bogus website almost instantly. Real people can even tag visual content such as pictures, this is something that computers have a really hard time doing.
What problems do Folksonomies have? Well one major problem is that they don’t guarantee completeness. You can’t tell if you have a complete result set or a partial result set. You also can’t tell if people are abusing the tagging system. Folksonomies have a higher level of trust and anyone using these techniques needs to be aware that they can be abused by users who self-tag their own pages and content to improve rankings.
As search engines develop I expect Folksonomies will become a larger part of the search engine algorithm. Using a combination of traditional editorial categories, web-crawling and folksonomies will produce better search results for the search engine of the future.