Twikicatalyzer – January 2010 Ideas
The fifth idea for my 365 Social Ideas is a Twitter- and Wikipedia-mashup idea (and some maps): Find out who’s talking about what on Twitter through Wikipedia categories. There are many sites for tracking Twitter trends , but they all require that you know what you’re looking for (or where you’re looking for it geographically). But what if you simply wanted to know what movies people are tweeting about? Or what about bands, musicians, flowers, four-legged mammals, financial institutions, politicians from Guadelupe or any other arbitrarily broad or narrow category of items?
For that purpose, Wikipedia categories seem very useful and given the openness of the Wikipedia philosophy, it would actually be quite simple. The site could be as simple as a single input field with input suggestions based on the Wikipedia category system. For example you could input “1950s in film” and maybe even intersect it with another category like “Best Picture Academy Award winners” and the resulting list (including all sub-category entries as well of course) would then be looked up in recent tweets on Twitter and the results would be displayed in different ways – including a list of most popular items, time-based breakdowns, geographic breakdowns, related links, most dominant profiles etc.
The technology behind it is quite simple and Wikipedia category search and even intersection has been seen in many different applications before. But using it for creating lists of items in a certain category to lookup in a different medium all-together is a new in my book.
Oh, and yes, the name used in the headline is quite stupid, agreed, but with Twitter and Wikipedia involved, Twiki-something seems appropriate.
Why?
Trending topics on Twitter are highly popular and the target of many services, lots of discussion and even some succesful spamming attempts. But the general trends are way to general to be interesting for analyzing what’s happening in a certain industry or area of interest. This application/service/mashup is a low-hanging fruit, and might carry some hype once deployed, but might not have a long-standing potential. I do however personally find it very interesting and believe that a lot of “social media experts”, trend researchers and marketeers will as well.
What’s next?
Do with this idea whatever you like – expand, implement, trash or forget. Just remember, that if you use it in anyway make sure to attribute me according to the Creative Commons Attribution 3.0 License, that all these 365 Social Ideas are published under.
Related posts:
Category: API, January 2010 Ideas, Wikipedia 4 comments »
January 6th, 2010 at 08:54
OK, a Wikipedia/twitter client? That is indeed low-hanging fruit. I’m not up-to speed on the latest version of MediaWiki, but the last time I looked, it didn’t really have an open architecture and an API, so I have to research that
January 6th, 2010 at 11:58
No, as far as I know (being an old dawiki admin and quite well-versed in the system back then) Wikipedia still does not have a live, queryable API, but probably a lot of static copies of it, that does. They are however only updated on a daily basis and could thus for the very newest things be a bit old. Wikipedia explicitly reserves the right to block clients assumed to be scraping content, but will only do so if they see extreme abuse that affects performance.
My idea was, that you would create a site, that would create their own cache of the entire Wikipedia category system and update this daily.
Even better and easier would be to use some of the existing category scanners – like CatScan (intersect of the above to cats is given here: http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php?wikilang=en&wikifam=.wikipedia.org&basecat=1950s+in+film&basedeep=10&mode=cs&tagcat=Best+Picture+Academy+Award+winners&tagdeep=3&go=Scan&format=html&userlang=en). But that requires, that they are externally queryable and it actually seems like it, as you can request CSV response like:
http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php?wikilang=en&wikifam=.wikipedia.org&basecat=1950s+in+film&basedeep=10&mode=cs&tagcat=Best+Picture+Academy+Award+winners&tagdeep=3&go=Scan&format=csv&userlang=en
Remember to check the tools terms of use though.
But another part of building this would be the type-ahead category input field, and that probably requires a full local copy of the entire category tree.
It actually seems so easy to do now, that I feel tempted to join ;)
January 6th, 2010 at 12:18
Thanks, great resources, and my thinking as well regarding the cache. I actually had a fat client in mind, but type ahead fields, even when using lowtech like the web, is np.
(sorry about the short answer, is on the cell)
January 23rd, 2010 at 08:36
Cool, there is actually some great ideas on here some of my subscribers may find this relevant, I will send them a link, many thanks.