Comments on: Twikicatalyzer – January 2010 Ideas http://www.barklund.org/blog/2010/01/05/twikicatalyzer-january-2010-idea/ work smarter when building current web trends Wed, 23 Nov 2011 11:48:49 +0000 hourly 1 http://wordpress.org/?v=3.0.4 By: #iwouldhatetobeyou http://www.barklund.org/blog/2010/01/05/twikicatalyzer-january-2010-idea/comment-page-1/#comment-101789 #iwouldhatetobeyou Sat, 23 Jan 2010 07:36:58 +0000 http://www.barklund.org/blog/?p=550#comment-101789 Cool, there is actually some great ideas on here some of my subscribers may find this relevant, I will send them a link, many thanks. Cool, there is actually some great ideas on here some of my subscribers may find this relevant, I will send them a link, many thanks.

]]> By: Kim Bach http://www.barklund.org/blog/2010/01/05/twikicatalyzer-january-2010-idea/comment-page-1/#comment-98664 Kim Bach Wed, 06 Jan 2010 11:18:43 +0000 http://www.barklund.org/blog/?p=550#comment-98664 Thanks, great resources, and my thinking as well regarding the cache. I actually had a fat client in mind, but type ahead fields, even when using lowtech like the web, is np. (sorry about the short answer, is on the cell) Thanks, great resources, and my thinking as well regarding the cache. I actually had a fat client in mind, but type ahead fields, even when using lowtech like the web, is np.
(sorry about the short answer, is on the cell)

]]> By: Barklund http://www.barklund.org/blog/2010/01/05/twikicatalyzer-january-2010-idea/comment-page-1/#comment-98660 Barklund Wed, 06 Jan 2010 10:58:39 +0000 http://www.barklund.org/blog/?p=550#comment-98660 No, as far as I know (being an old dawiki admin and quite well-versed in the system back then) Wikipedia still does not have a live, queryable API, but probably a lot of static copies of it, that does. They are however only updated on a daily basis and could thus for the very newest things be a bit old. Wikipedia explicitly reserves the right to block clients assumed to be scraping content, but will only do so if they see extreme abuse that affects performance. My idea was, that you would create a site, that would create their own cache of the entire Wikipedia category system and update this daily. Even better and easier would be to use some of the existing category scanners - like CatScan (intersect of the above to cats is given here: http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php?wikilang=en&wikifam=.wikipedia.org&basecat=1950s+in+film&basedeep=10&mode=cs&tagcat=Best+Picture+Academy+Award+winners&tagdeep=3&go=Scan&format=html&userlang=en). But that requires, that they are externally queryable and it actually seems like it, as you can request CSV response like: http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php?wikilang=en&wikifam=.wikipedia.org&basecat=1950s+in+film&basedeep=10&mode=cs&tagcat=Best+Picture+Academy+Award+winners&tagdeep=3&go=Scan&format=csv&userlang=en Remember to check the tools terms of use though. But another part of building this would be the type-ahead category input field, and that probably requires a full local copy of the entire category tree. It actually seems so easy to do now, that I feel tempted to join ;) No, as far as I know (being an old dawiki admin and quite well-versed in the system back then) Wikipedia still does not have a live, queryable API, but probably a lot of static copies of it, that does. They are however only updated on a daily basis and could thus for the very newest things be a bit old. Wikipedia explicitly reserves the right to block clients assumed to be scraping content, but will only do so if they see extreme abuse that affects performance.

My idea was, that you would create a site, that would create their own cache of the entire Wikipedia category system and update this daily.

Even better and easier would be to use some of the existing category scanners – like CatScan (intersect of the above to cats is given here: http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php?wikilang=en&wikifam=.wikipedia.org&basecat=1950s+in+film&basedeep=10&mode=cs&tagcat=Best+Picture+Academy+Award+winners&tagdeep=3&go=Scan&format=html&userlang=en). But that requires, that they are externally queryable and it actually seems like it, as you can request CSV response like:

http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php?wikilang=en&wikifam=.wikipedia.org&basecat=1950s+in+film&basedeep=10&mode=cs&tagcat=Best+Picture+Academy+Award+winners&tagdeep=3&go=Scan&format=csv&userlang=en

Remember to check the tools terms of use though.

But another part of building this would be the type-ahead category input field, and that probably requires a full local copy of the entire category tree.

It actually seems so easy to do now, that I feel tempted to join ;)

]]> By: Kim Bach http://www.barklund.org/blog/2010/01/05/twikicatalyzer-january-2010-idea/comment-page-1/#comment-98627 Kim Bach Wed, 06 Jan 2010 07:54:31 +0000 http://www.barklund.org/blog/?p=550#comment-98627 OK, a Wikipedia/twitter client? That is indeed low-hanging fruit. I'm not up-to speed on the latest version of MediaWiki, but the last time I looked, it didn't really have an open architecture and an API, so I have to research that OK, a Wikipedia/twitter client? That is indeed low-hanging fruit. I’m not up-to speed on the latest version of MediaWiki, but the last time I looked, it didn’t really have an open architecture and an API, so I have to research that

]]>