Thursday 4 June 2009

Searching for meaning.

If the internet were a giant book, containing all of human knowledge, it would surely have a search engine as it’s index. And what an index that would be! With the seemingly infinite variety of data and the breadth and the depth of information on the internet, an index this big would probably fill the largest of libraries, on its own.
When you think about it, the accessibility and indeed the popularity of the internet, is due of the humble search engine. I mean if it weren’t for search engines tracking all this information, finding relevant information, would be like finding a needle in a haystack.
But we’re rarely satisfied with what we have, so merely getting pages of results to our queries is no longer enough. What we want now is to get only information that is highly relevant. This is where semantic search comes in.
“Semantic” just means meaningful, how do I know? I ‘Googled’ it! So what does ‘meaningful search’ mean then? Of course it has different meanings to different people, but most people agree that it should be; a search that understand simple ‘who, how, when, where and what’ questions, and that it should be able to provide clear,simple,unambiguous answers. For example you if you typed, “how old is Harrison Ford?’, or even ‘who is bill clinton married to?’, you might get the answers like ‘66 years old’ or ‘Hillary Rodham Clinton’.
You might be surprised to learn that Google have already introduced this technology into their search engine, (if somewhat quietly). Yahoo have also been working on their own semantic technology with a project called ‘Search monkey’. The Yahoo solution works with microformats to allow site owners, designers and users to hide additional information beneath the page, making it easier for search engines to interpret the context of a page and return more relevant results. Microsoft also launched 'Bing.com' their replacement to 'live search' previously known as project ‘Kino’ and possibly including some 'Powerset' Technology.
But is searching semantically enough? After all semantic searches will still only provide information that already exists on the internet. What if we want, answers to questions, where that particular information doesn’t currently exist. To do this firstly we would need to understand the question itself, then we would need to understand the data that could use to answer the question and finally we would need to be able to calculate an appropriate answers.

You’d be forgiven therefore, if you thought that this was the stuff of science fiction, but you’d be wrong. In mid May, Stephen Wolfram released ‘Wolfram Alpha’, he describes this as a ‘computational knowledge engine’. It’s limited to mainly scientific data (in its own database) at the moment, but it could change everything.

This post text was originally wtitten for the technotes column in the journal on 4th June 2009. Many Thanks to Lewis Harrison for the editting of the final piece.

blog comments powered by Disqus