Turn (Closed) Content Into an API – January 2010 Ideas
The twenty-eighth idea for my 365 social ideas is about the open web and about “forcing” classic websites to export their data. Imagine sites with lots of useful information, that is frequently updated, but is hidden away behind forms, in PDF’s or in hard-to-scrape tables. Then imagine a website, where you could provide this address, and give it some guidance as to how to input data in forms and how to interpret the results. And then imagine, that this website would act as a proxy with this interaction described as a simple, queryable API and then behind the scenes would fetch data from the original website.
There are of course a lot of legal issues, and such a service should probably obey robots.txt. And there should be a very clear opt-out possibility for websites that have been targeted in user-contributed additions – maybe even moderation before the services are publicly available. It should not be seen as an attack vector, as a proxy method for illegal purposes or anything like that, but simply as a way of “helping” free information to be used freely.
Why?
A lot of data on the web is hidden in the so-called “deep web” behind forms or in inaccessible parts of websites. With a service like the above-mentioned, this would suddenly not only become visible and indexable, but even queryable.
What’s next?
Do with this idea whatever you like – expand, implement, trash or forget. Just remember, that if you use it in anyway make sure to attribute me according to the Creative Commons Attribution 3.0 License, that all these 365 Social Ideas are published under.
Related posts:
Category: API, January 2010 Ideas, Online Rights, Security Comment »