For that purpose, Wikipedia categories seem very useful and given the openness of the Wikipedia philosophy, it would actually be quite simple. The site could be as simple as a single input field with input suggestions based on the Wikipedia category system. For example you could input “1950s in film” and maybe even intersect it with another category like “Best Picture Academy Award winners” and the resulting list (including all sub-category entries as well of course) would then be looked up in recent tweets on Twitter and the results would be displayed in different ways – including a list of most popular items, time-based breakdowns, geographic breakdowns, related links, most dominant profiles etc.

The technology behind it is quite simple and Wikipedia category search and even intersection has been seen in many different applications before. But using it for creating lists of items in a certain category to lookup in a different medium all-together is a new in my book.

Oh, and yes, the name used in the headline is quite stupid, agreed, but with Twitter and Wikipedia involved, Twiki-something seems appropriate.

Why?

Trending topics on Twitter are highly popular and the target of many services, lots of discussion and even some succesful spamming attempts. But the general trends are way to general to be interesting for analyzing what’s happening in a certain industry or area of interest. This application/service/mashup is a low-hanging fruit, and might carry some hype once deployed, but might not have a long-standing potential. I do however personally find it very interesting and believe that a lot of “social media experts”, trend researchers and marketeers will as well.

What’s next?

Do with this idea whatever you like – expand, implement, trash or forget. Just remember, that if you use it in anyway make sure to attribute me according to the Creative Commons Attribution 3.0 License, that all these 365 Social Ideas are published under.

Related posts:

1. Twitter Follow Organizer – January 2010 Ideas
2. Game of Life – January 2010 Ideas
3. Twadio – January 2010 Ideas

After reading up on UK copyright law, my judgment is unclear – but mostly in favor of the NPG and thus against Wikimedia.

The CDPA of 1988 clearly states in the very first paragraph:

1. Copyright and copyright works
   1. Copyright is a property right which subsists in accordance with this Part in the following descriptions of work—
      1. original literary, dramatic, musical or artistic works,
      2. [...]

To me, one crucial part is the word “original”. This removes the possibility of claiming, that merely taking a photograph of another work is copyrightable in itself. The law goes on to specify what each of the four here-described works are, and it is clear, that taking a photo is an artistic work (defined in §4 sec. 1.a), but it is not an original work (in my opinion), when it is a photograph of a painting.

But the manipulation done by the conservationists when digitizing the photos includes a lot of work, which is definitely craftsmanship, but if deemed as artistic craftsmanship, it is in itself covered by the CDPA in §4 section 1.c:

4. Artistic works
   1. In this Part “artistic work” means—
      2. [...]
      3. a work of artistic craftsmanship.

My conclusion is thus, that it is irrelevant whether it is a derivative work or not (non-original works are never copyrightable) – the matter of the case is, whether the work of conservationists is artistic craftsmanship or not. And I’m leaning towards a “yes”, but I’m pretty sure the case cannot be settled without a court ruling, as it is clearly a case of interpretation. You can argue that a conservationist is merely a scientist, not an artist – but even scientific papers and articles are subject to copyright, and thus I believe, that the work of a conservationist is subject to copyright given that it is artistic craftsmanship and/or original scientific work.

Related posts:

1. How SnapABug works – and what they should do
2. The iPhone developer boycott in the works

On IRC, the Danish Wikipedia channel on freenode, we have a bot running (built on Linky again built on PircBot). This bot’s primary purpose is to extend socalled wikilinks. That is, when someone writes Someone wrote silly stuff in the [[USA]] article again the bot replies http://da.wikipedia.org/wiki/USA – it expands the bracket-style links to proper URL’s.

The problem was the character encodings. Some use UTF-8 in this channel and others use ISO 8859-1. And how can you make the bot expand links for the Danish term [[Kødpålæg]] when written in either of the two encodings? The correct URL for this word is K%C3%B8dp%C3%A5l%C3%A6g The built-in UTF-8-decoder in Java will replace the “bad characters” with the Unicode replacement character U+FFFD, and thus the previous term written sent by a ISO 8859-1 client with Linky in UTF-8 mode would become K%EF%BF%BDdp%EF%BF%BDl%EF%BF%BDg, and in the reverse situation with Linky in ISO 8859-1 mode, the UTF-8 clients messages would be interpreted to K%C3%83%C2%B8dp%C3%83%C2%A5l%C3%83%C2%A6g. Both are very wrong.

The solution is to have Linky run in UTF-8 mode, but when incorrect byte sequences occur, don’t replace them with the (proper) replacement character, but instead translate them using ISO 8859-1. And this is exactly, what I have created. A jar-file with a CharsetProvider providing a X-UTF-8-Failover-charset that simply does this. Then I added this jar to the classpath when running Linky and set the charset of Linky to be X-UTF-8-Failover – and it works!

The hard parts of this were:

1. How do I decode UTF-8 properly?
2. How do I determine bad sequences?
3. How do I decode these differently instead?
4. How do I create a charset that Java can use?
5. How do I link this so Linky will use it?

Well, questions 1 and 2 I had some prior knowledge about, but reading specs was still required. Question 3 was very simple once I realised, that the characters in UTF-8 below U+0100 is exactly the same as in ISO 8859-1. Question 4 was answered partly by reading the documentation for CharsetProvider and partly by this nice overview of Java and encodings. The fifth question was a simple matter of classpath.

To get to the main part of the whole deal, the decodeLoop-implementation for X-UTF-8-Failover looks like this:

protected CoderResult decodeLoop(ByteBuffer in, CharBuffer out) {
	int inPos = in.position();
	try {
		while (in.hasRemaining()) {
			char c;
			byte b1 = in.get();
			int highNibble = (b1 >> 4) & 0xF;
 
			switch (highNibble) {
			case 0:
			case 1:
			case 2:
			case 3:
			case 4:
			case 5:
			case 6:
			case 7:
				if (out.remaining() < 1)
					return CoderResult.OVERFLOW;
				out.put((char) b1);
				inPos = in.position();
				break;
 
			case 0xC:
			case 0xD:
				byte b2;
				if (in.remaining() < 1)
					return CoderResult.UNDERFLOW;
				if (out.remaining() < 1)
					return CoderResult.OVERFLOW;
				if (!isContinuation(b2 = in.get())) {
					// put second char back by rewinding, putting and re-rewinding
					in.position(in.position()-1);
					in.put(b2);
					in.position(in.position()-1);
					// update last read legal position
					inPos = in.position();
					// append byte directly used
					out.append((char) (b1 & 0xFF));
					// break switch
					break;
				}
				c = (char) (((b1 & 0x1F) << 6) | (b2 & 0x3F));
				// check that we had the shortest encoding
				if (c <= 0x7F)
					return CoderResult.malformedForLength(2);
				out.put(c);
				inPos = in.position();
				break;
 
			case 0xE:
				byte b3;
				if (in.remaining() < 2)
					return CoderResult.UNDERFLOW;
				if (out.remaining() < 1)
					return CoderResult.OVERFLOW;
				if (!isContinuation(b2 = in.get())) {
					// put second char back by rewinding, putting and re-rewinding
					in.position(in.position()-1);
					in.put(b2);
					in.position(in.position()-1);
					// update last read legal position
					inPos = in.position();
					// append byte directly used
					out.append((char) (b1 & 0xFF));
					// break switch
					break;
				}
				if (!isContinuation(b3 = in.get())) {
					// put third char back by rewinding, putting and re-rewinding
					in.position(in.position()-1);
					in.put(b3);
					in.position(in.position()-1);
					// update last read legal position
					inPos = in.position();
					// append first and second byte directly used
					out.append((char) (b1 & 0xFF));
					out.append((char) (b2 & 0xFF));
					// break switch
					break;
				}
				c = (char) (((b1 & 0x0F) << 12) | ((b2 & 0x3F) << 6) | (b3 & 0x3F));
				// check that we had the shortest encoding
				if (c <= 0x7FF)
					return CoderResult.malformedForLength(3);
				out.put(c);
				inPos = in.position();
				break;
 
			default:
				// parse as latin 1
				out.append((char) (b1 & 0xFF));
				inPos = in.position();
				break;
			}
		}
 
		return CoderResult.UNDERFLOW;
	} finally {
		// In case we did a get(), then encountered an error, reset the
		// position to before the error.  If there was no error, this
		// will benignly reset the position to the value it already has.
		in.position(inPos);
	}
}

Complete source for this can be download here under the regular license.

No related posts.