Searching for meaning.

If the internet were a giant book, containing all of human knowledge, it would surely have a search engine as it’s index. And what an index that would be! With the seemingly infinite variety of data and the breadth and the depth of information on the internet, an index this big would probably fill the largest of libraries, on its own.
When you think about it, the accessibility and indeed the popularity of the internet, is due of the humble search engine. I mean if it weren’t for search engines tracking all this information, finding relevant information, would be like finding a needle in a haystack.
But we’re rarely satisfied with what we have, so merely getting pages of results to our queries is no longer enough. What we want now is to get only information that is highly relevant. This is where semantic search comes in.
“Semantic” just means meaningful, how do I know? I ‘Googled’ it! So what does ‘meaningful search’ mean then? Of course it has different meanings to different people, but most people agree that it should be; a search that understand simple ‘who, how, when, where and what’ questions, and that it should be able to provide clear,simple,unambiguous answers. For example you if you typed, “how old is Harrison Ford?’, or even ‘who is bill clinton married to?’, you might get the answers like ‘66 years old’ or ‘Hillary Rodham Clinton’.
You might be surprised to learn that Google have already introduced this technology into their search engine, (if somewhat quietly). Yahoo have also been working on their own semantic technology with a project called ‘Search monkey’. The Yahoo solution works with microformats to allow site owners, designers and users to hide additional information beneath the page, making it easier for search engines to interpret the context of a page and return more relevant results. Microsoft also launched 'Bing.com' their replacement to 'live search' previously known as project ‘Kino’ and possibly including some 'Powerset' Technology.
But is searching semantically enough? After all semantic searches will still only provide information that already exists on the internet. What if we want, answers to questions, where that particular information doesn’t currently exist. To do this firstly we would need to understand the question itself, then we would need to understand the data that could use to answer the question and finally we would need to be able to calculate an appropriate answers.

You’d be forgiven therefore, if you thought that this was the stuff of science fiction, but you’d be wrong. In mid May, Stephen Wolfram released ‘Wolfram Alpha’, he describes this as a ‘computational knowledge engine’. It’s limited to mainly scientific data (in its own database) at the moment, but it could change everything.

This post text was originally wtitten for the technotes column in the journal on 4th June 2009. Many Thanks to Lewis Harrison for the editting of the final piece.

Releasing a WAVE of new possibilities.

Google never cease to astonish me with their innovation.  In a month when 'Wolfram Alpha' and 'Microsoft Bing' filled the technology headlines, and people were talking about Twitter and the way that real time search was changing everything, they announce WAVE and completely turned the whole communication model on its head.

In typical Google style, they keep there best until last, waiting until the second day of the Google Developer Conference in the Moscone Center, San Francisco on May 27 - 28, 2009, to show us WAVE. 

So what's so good about WAVE and why did i say it will turn the communications model on its head? Up until now we have tried to imitate none-electronic world (ie the paper world) with our online communications technologies, sending - then receiving, storing documents in folders, working in structured linear patterns publishing the finished product without showing how we got there, keeping separate information in separate places, reading-writing-sharing-collaborating. All this could soon change. Wave allows us to work together in an altogether different way, working in real time to seeing content as its typed, utilizing richly formatted text, photos, videos, maps and other tools.

And its not just the way that we communicate in the sense of instant messaging and text, that is going to change its also things like blogs and websites, imaging being able to collaboratively leave comments to a blog post, where you could use text,images, movies, maps and more to make your point, or a pole where you could discuss your opinions in real time with other users, and that is only a beginning, the implications for the way we watch movies, sports or tv, or even the way that we play games or do puzzles on line could be radically different, and after a day, we haven't even started to explore the possibilities. Imagine the way that mass collaborations may work with science or the arts, the way that we conduct research or that we learn. Its like Lars and Jens Rasmussen, have opened a flood gate on web development and release a wave of new possibilities

Part of what makes what the Google Wave team have done special, is that its all open source, and its all possible in the browser (yes its html 5 so you will need a modern browser, but one the whole it doesn't need any special add ins). So we're likely to see the technology being pushed to all sorts of extremes with developers coming up with all sorts of clever and innovative ways to utilize this technology, in other words we ain't seen nothin yet!

Google have always been great innovators, but they've never been satisfied with that, they seem determined to rewrite every rules and make us rethink everything we thought we knew. 

Related posts: Google WAVE,  Will google wave goodbye to twitter?

5 things to do if you didn't get Thinking Digital tickets!

Its hard to believe that this years Thinking Digital Conference is now less than a month away. The event, now in its 4th year, is rapidly shaping up to be amongst the best tech conferences around, not only nationally but internationally. One of the things that really stands out for me is how much the speakers themselves seem to enjoy the event.

Thinking Digital in Under Three Minutes from Herbert Kim on Vimeo.

Not surprisingly the tickets sold out pretty quickly. So as Herb Kim and and his team, put the final touches together, what can you do if you didn't manage to get a ticket, here are my top 5 suggestions;

1. Come to Newcastle anyway and hang around hotel bars hoping to bump into the speakers, although you may have to watch out for hotel security throwing you out.
2. Get yourself on Amazon and order copies of the speakers books and start speed reading.
3. Watch some of last years talks on youtube and hope that nothings changed in the past 12 months.
4. Follow the blogs and tweets from the event, wishing that you'd found a way to get a ticket.
5. Send a strongly worded email to the boss asking why you're not there and demanding a ticket for next years event.

In all seriousness the Thinking Digital Conference is amongst the best conferences around, and i'm sure that those of you that did manage to get tickets will have a great time, meet lots of fantastic people and go home with a head full of great ideas and enthusiasm. If you really haven't been lucky enough to get tickets for this years event and would like to attend i believe there is a reserve list (although i have no idea how long it is) and there is always next year.

CloudCamp North-East England

Newcastle University's Beehive building played host to the first UK Cloudcamp outside on London on Tuesday 24th March. Officially an unconference, entry was free and anyone that wanted to speak had an opportunity to take a slot, it also followed the great tradition of Barcamps with free beer and pizza and plenty of time for networking. A little more structured than a typical unconference, starting with a series of lightening talks and moving on to a panel discussion.


After a brief introduction Justin Souter kicked off the lighting talks with 'an introduction to cloud computing', Steve Tron of Knowledge IT talked about 'public and fedorated clouds', Tony Lucas, CEO of XCalibre talked about 'whether there is a need for private clouds', Chris Purrington from CohesiveFT covered 'taking control in the cloud', Arvid Fossen of Aserver went on to talk about 'ready to use clouds',
Ross Cooney, founder of EmailCloud did a spot titled ‘bootstrap and transition, cloud computing to get your business started’, Alex Heneveld, CTO of CloudSoft covered 'cloud routemaster’, while Steve Caughey, CEO of Arjuna took ‘how to obtain Quality Of Service from a cloud?’ leaving Duncan Malcolm of EveryCity talking 'cloud1 versus cloud2' and Stewart Townsend from SUN rounded off the lightening talks.

The lightening talks were followed by a lively panel discussion chaired by Ross Cooney, where the 80 or so delegates got to pick the brains of the panel. The discussion covering everything from standards and data portability, to how do you tell your IT team that their jobs no longer exist and questions like how do businesses cope with a utility billing approach to cost and what happens if your provider goes bust.

All in all, it was very successful evening and I know I had a great time and learned a lot. While I found all of it interesting Ross's talk was one of the most compelling. Ross's company, Rozmic is relatively small, a handful of employees providing services for email and messaging, in a traditional model they would have a large building housing enough servers to cope with maximum usage (117 servers), a team of engineers, a huge internet pipe and power facilities. This would have had to be purchased and set up, and the maintenace and upkeep would be quite a task. With cloud computing he's paying per server, per hour and automatically turning it off and on as required, so during the night when demand is low he's only paying for 1 or 2 machines, during the day as demand goes up, he maintains performance by turning on more servers. He's only paying for what he uses and paying as he goes, so cash flow is managable and as the servers are virtual maintenace is minimal, giving his engineers time to streamline the code and make it even more efficient. What this means is, he can afford to use the technology he wants to use and can grow his business, without large risky upfront investments.

Having thought about it since i'm still not sure that I fully understand the implications of moving your IT into the cloud. Here are a few of the things that occured to me:

If you are deploying all your data to the cloud, and spinning up server instances all over the place, how does this affect the networking model? i.e. are you still using active directory to control your users, computers, sites and services? And where does anti virus and maleware fit in? Then, there is the whole area of licencing, if your creating hundreds of new servers on the fly how are they being licensed does it still use the per seat or per server approach, even if your not using microsoft servers and services surely there'll be licenses for anti virus, back up etc.

When we talk about "the cloud" are we talking about a single cloud or many fluffy little clouds, ie would i use amazon to store my files and cloudmail to provide my email and someone else to supply my sql and someone else for my cms and if this is the case how do i back all of this up, do i have lots and lots of little back up all over this cloud.

And the final thing to leave you with is do we have enough people with the skills to do all this? As an IT manager of some years, i have a very good understanding of core It, backup, disaster recovery, and a little knowledge of Api's and web development,but i'm not entirely sure that most IT staff would have all of the skills required to do all of the tricky bits to make this seemless to their end users, and i don't think many sme's could afford to get an engineer in every time it needs a tweek.

Resources: Introduction to cloud computing by Souter Consulting, Justin's blog post (with transcripts of the questions and answers), Ross's blog post, David coxon's photos.

What's after Web 2.0?

Throughout January and February Justin Souter has published a guide to Web 2.0 on b.daily, explaining - what, where, when, why, how and who's. Justin Asked me to take a look and give him some feedback, and I want to expand upon it and take a look at what the future might include.

Web 2.0 marked a paradigm shift in the internet: technological advances converged, and coincided with a fall in costs and a change in public attitudes that is not going to happen again for a while.

What we are far more likely to see in the future are faster, smaller changes in the way that we view and use the internet (and technology in general).

This trend is, in part, because of Web 2.0. Developers have become used to leveraging user generated content created in small, loosely connected pockets all over the internet and users are demanding sites and services that can react quickly to their changing needs.

With companies like Google, Facebook and Twitter opening up access to their services, it's becoming increasingly possible for bedroom developers to make, market and sell viable products online. It's likely that this means more mobile web. Already in Africa, Asia and other parts of the world there are more mobile users than desktop users. It's likely to mean more internet-enabled devices (cameras, TVs, watches, cars, household items, fridges, game consoles, electronic photo frames, even houses themselves).

It's likely to mean more unified communications (intelligently mixing and matching email, messaging, chat, mobiles and landlines). It's likely that the environment and green computing will continue to grow in importance and that cloud computing and virtualisation will be huge. We're almost certainly likely to see more social networking, and in particular growing internet global communities (I network with as many people internationally as I do locally already).

I couldn't finish without mentioning search, and the quest for semantic (clever) search engines. Google are obviously the clear market leader, but we are also seeing a move towards searching things like twitter and social bookmarking sites, and wanting more graphical search results.

We've been talking about the future though, and these are only my ideas on what may be coming. I'd really love is to hear your ideas on this and on what you think is going to be the next big thing, or just as importantly what's not. So please leave a comment.