Why?

Sharing snapshots of what you see currently is difficult. It is a lot easier on a Mac than on Windows, though Windows 7 has made it easier, but sharing it on the web still requires that you go to some website, browse for the image locally and upload in order to obtain a shareable url. If you had a simple bookmarklet as explained above, this process would be much easier.

What’s next?

Do with this idea whatever you like – expand, implement, trash or forget. Just remember, that if you use it in anyway make sure to attribute me according to the Creative Commons Attribution 3.0 License, that all these 365 Social Ideas are published under.

Related posts:

1. Private Comment Sharing – January 2010 Ideas
2. Offline News Aggregator – January 2010 Ideas
3. Social Traffic Analyzer – January 2010 Ideas

SnapABug did the right thing, the only thing they could do. Untrusted (unsigned) code lives in a special sandbox and cannot use functions outside of this sandbox. The end user can change how much this sandbox has access to, but it is not trivial and the normal user would never touch that with a pitch fork. Trusted code lives in another sandbox with almost infinite possibilities. The end user can limit this sandbox too, but again almost no end user does that.

The problem? How is code deemed to be trusted code? Code is trusted, if the publisher is trusted, and the validity of the publisher is guaranteed through the digital signature. The signature is automatically verified by the Java client, and the end user is then asked to do the final step – the Java client asks the end user to trust the code:

We know this code has been written by this person (even though we don’t know what the code does). We know this person/company is, who (s)he says (s)he is. You can therefore safely trust the authenticity of this person. Do you trust that this person has written safe code?

First of all, this is a strange question to ask any end user. Secondly, that if the actual question was written as above, it would have been much better as it would assume that the person has written safe code, but would let the end user decide – instead the question is written in a form that almost suggests, that the end user tries to illegally gain access to your computer by asking:

An applet from “domain” is requesting unrestricted access to your computer

The SnapABug applet trust question

And below that in smaller letters is says that the digital signature has been verified (whatever that means to Average Joe). I am aware, that this might be the more specific question asked by my Java client/browser combination, as Sun seems to indicate, that the dialog looks different on other systems.

In any case: that question is stupid. Would the normal end user install programs, if the standard windows installation flow included the question:

When installing this application, the application could potentially install spamware, adware, take over your computer or copy your banking information. Do you wish to proceed?

Let’s just say that adoptions of small programs from small developers would be a lot lower (which might be a good thing to prevent all those installing spamware because they think they have a virus or silly things like that, but in the long run it would probably be a bad thing). Asking such a question is stupid. This question really should be re-phrased in a way to clearly indicate, that the user should assume that this is a safe, trusted application, but the user has the opportunity to deny access anyway.

Secondly, (trusted) applications should have a way of asking for access to only what they need. The digital signature is simply for verification of origin, but the actual access level needed should be separate from that. As I asked in my previous post: why should the SnapABug applet have access to files on my computer? And they don’t need it, in fact I’m pretty sure, that if they could avoid having that access, they would rather do that. But they can’t.

Thus SnapABug is not to blame – if anyone, Sun is.

Related posts:

1. How SnapABug works – and what they should do
2. If WordPress automatic update fails “silently”…
3. Password-friendly Website Certificate – January 2010 Ideas

Well, I delved into the application and found the answers – they could of course have done with a more restrictive permission.

First I found the JavaScript file. The interesting part here was to find, that they actually inserted a java-applet in the corner of the application (in the startWebCapture-function):

startWebCapture:function() {
    	document.getElementById('SnapABug_Applet').style.display = 'block';
    	document.getElementById('SnapABug_Applet').innerHTML = 
	    '<applet style="position:absolute;background-position-x:0px;background-position-y:0px;display:inline;font-size:2px;width:300px;height:2px;" name="SnapABug_Applet" code="webCaptureApplet.WebCaptureApplet.class" codebase="' + baseURL + '/" archive="sWebCaptureApplet.jar" width="200" height="2" mayscript>'
        + '<param name="h" value="' + this.getWindowHeight() + '">'
        + '<param name="w" value="' + this.getWindowWidth() + '">'
        + '<param name="c" value="' + caseId + '">'
        + '<param name="d" value="300">'
        + '</applet>';
    }

This applet (JAR file) is passed the viewport size and a case ID (used for sending the screenshot directly to the server). The source for this applet is easily accessible using e.g. JD.

The interesting part here is how they actually find the offset of the viewport. Getting a screenshot of all the available screen devices is quite easily managed through the GraphicsEnvironment class iterating through all screen GraphicsDevice‘s to make it work on multi-screen setups. But finding the browser-window on the screenshot bitmap is a bit tricky – the browser chrome can look in all different sorts of ways, the only thing that we can be sure of is, that the viewport is rectangular and has the size given to the applet by JavaScript. If we could find just one of the corners of the viewport… Well, they solve this in a very low-tech way by via the above JavaScript embedding the Java Applet in the top-left corner at 2*200px and then in the paint(Graphics g)-function of the applet draw a certain pattern in that area:

  ...
  BigInteger bi = new BigInteger("11010100110010110011", 2);
  ...
  public void paint(Graphics g)
  {
    System.out.println("paint: drawing corner");
    drawCornerCode(g);
  ...
  private void drawCornerCode(Graphics g)
  {
    g.clearRect(0, 0, 100, 10);
    for (int i = 0; i < 20; ++i) {
      if (this.bi.testBit(19 - i))
        g.setColor(Color.black);
      else {
        g.setColor(Color.white);
      }
      g.drawLine(i * 3, 0, i * 3, 10);
      g.drawLine(i * 3 + 1, 0, i * 3 + 1, 10);
      g.drawLine(i * 3 + 2, 0, i * 3 + 2, 10);
    }
  }

This draws a black and white striped pattern in the top left corner of the browser which functions somewhat like a bar code. Then, when JavaScript asks the applet to create a screenshot, the applet runs through every pixel of every screen and sees, if from that pixel the bar code follows:

      for (int j = 0; (j < devices.length) && (!(detected)); ++j) {
        GraphicsDevice device = devices[j];
        GraphicsConfiguration config = device.getDefaultConfiguration();
        Rectangle bounds = config.getBounds();
        ...
          screenshot = rbt.createScreenCapture(bounds);
        ...
        for (int y = 0; (y < screenshot.getHeight()) && (!(detected)); ++y) {
          for (int x = 0; (x < screenshot.getWidth() - 21) && (!(detected)); x += 2)
          {
            if (screenshot.getRGB(x, y) != Color.black.getRGB()) {
              continue;
            }
            for (int i = 0; i < 20; ++i) {
              if (((this.bi.testBit(19 - i)) && (screenshot.getRGB(x + i * 3, y) == Color.black.getRGB())) || (
                (!(this.bi.testBit(19 - i))) && (screenshot.getRGB(x + i * 3, y) == Color.white.getRGB())))
              {
                detected = true;
              } else {
                detected = false;
                break;
              }
            }
            if (detected) {
              System.out.println("detected x: " + x + " y: " + y);
              detectedX = x;
              detectedY = y;
            }
          }

This is actually quite brilliant, but of course not a revolution – just a smart simple way of using what’s available.

What is the problem then? Unrestricted access is the problem! I never allow unrestricted access to any applet unless I can’t do without it and trust the publisher. For this tool, neither criteria is fulfilled. After having checked the source code, I have allowed the applet, but not before – unrestricted access is simply way more than they should ever have. They need access for two things:

1. AWTPermission – access to the screens (which also gives access to create frameless windows and access to the visible state of all other programs running).
2. SocketPermission – access to cross-domain posting of the final image.

Of these permissions, only the first one is really necessary. The latter could be solved by requiring implementers to have a script locally on their own domain, that could process submitted images or simply forward them.

If the applet only asked permission to these two – or even better only AWTPermission – it would be way more trustworthy. As Sun says about the AllPermission:

java.security.AllPermission specifies all permissions in the system for all possible targets and actions. This permission should be used only during testing because it grants permission to run with all security restrictions disabled as if there were no security manager.

Only during testing! Never in deployed applets from an unknown company expecting major adoption across third-party websites. The permission, that I particularly don’t like any applet not needing it to have is FilePermission. They should not be able to read any files on my computer or write any – they don’t need it!

If it does get implemented on many sites, this jar file will then be very very interesting to phishers and scammers, as they could use this (if they replaced the file with their own) to infiltrate many computers with key-loggers, malware and anything else. This is not the case, as the applet is digitally signed, so modifying would require re-signing it, which would require re-authorizing it by end users.

But nevertheless – a very interesting feedback mechanism, that definitely has potential.

Source-code published under fair use. All rights belong to Timzon and only published here as-is without any warranty, obligations or any claims of copyright on my behalf.

Related posts:

1. Why signed applet trust is a stupid question (and why SnapABug is not to blame)
2. The iPhone developer boycott in the works
3. Why UK copyright law does extend copyright to the digitization of public domain works

On IRC, the Danish Wikipedia channel on freenode, we have a bot running (built on Linky again built on PircBot). This bot’s primary purpose is to extend socalled wikilinks. That is, when someone writes Someone wrote silly stuff in the [[USA]] article again the bot replies http://da.wikipedia.org/wiki/USA – it expands the bracket-style links to proper URL’s.

The problem was the character encodings. Some use UTF-8 in this channel and others use ISO 8859-1. And how can you make the bot expand links for the Danish term [[Kødpålæg]] when written in either of the two encodings? The correct URL for this word is K%C3%B8dp%C3%A5l%C3%A6g The built-in UTF-8-decoder in Java will replace the “bad characters” with the Unicode replacement character U+FFFD, and thus the previous term written sent by a ISO 8859-1 client with Linky in UTF-8 mode would become K%EF%BF%BDdp%EF%BF%BDl%EF%BF%BDg, and in the reverse situation with Linky in ISO 8859-1 mode, the UTF-8 clients messages would be interpreted to K%C3%83%C2%B8dp%C3%83%C2%A5l%C3%83%C2%A6g. Both are very wrong.

The solution is to have Linky run in UTF-8 mode, but when incorrect byte sequences occur, don’t replace them with the (proper) replacement character, but instead translate them using ISO 8859-1. And this is exactly, what I have created. A jar-file with a CharsetProvider providing a X-UTF-8-Failover-charset that simply does this. Then I added this jar to the classpath when running Linky and set the charset of Linky to be X-UTF-8-Failover – and it works!

The hard parts of this were:

1. How do I decode UTF-8 properly?
2. How do I determine bad sequences?
3. How do I decode these differently instead?
4. How do I create a charset that Java can use?
5. How do I link this so Linky will use it?

Well, questions 1 and 2 I had some prior knowledge about, but reading specs was still required. Question 3 was very simple once I realised, that the characters in UTF-8 below U+0100 is exactly the same as in ISO 8859-1. Question 4 was answered partly by reading the documentation for CharsetProvider and partly by this nice overview of Java and encodings. The fifth question was a simple matter of classpath.

To get to the main part of the whole deal, the decodeLoop-implementation for X-UTF-8-Failover looks like this:

protected CoderResult decodeLoop(ByteBuffer in, CharBuffer out) {
	int inPos = in.position();
	try {
		while (in.hasRemaining()) {
			char c;
			byte b1 = in.get();
			int highNibble = (b1 >> 4) & 0xF;
 
			switch (highNibble) {
			case 0:
			case 1:
			case 2:
			case 3:
			case 4:
			case 5:
			case 6:
			case 7:
				if (out.remaining() < 1)
					return CoderResult.OVERFLOW;
				out.put((char) b1);
				inPos = in.position();
				break;
 
			case 0xC:
			case 0xD:
				byte b2;
				if (in.remaining() < 1)
					return CoderResult.UNDERFLOW;
				if (out.remaining() < 1)
					return CoderResult.OVERFLOW;
				if (!isContinuation(b2 = in.get())) {
					// put second char back by rewinding, putting and re-rewinding
					in.position(in.position()-1);
					in.put(b2);
					in.position(in.position()-1);
					// update last read legal position
					inPos = in.position();
					// append byte directly used
					out.append((char) (b1 & 0xFF));
					// break switch
					break;
				}
				c = (char) (((b1 & 0x1F) << 6) | (b2 & 0x3F));
				// check that we had the shortest encoding
				if (c <= 0x7F)
					return CoderResult.malformedForLength(2);
				out.put(c);
				inPos = in.position();
				break;
 
			case 0xE:
				byte b3;
				if (in.remaining() < 2)
					return CoderResult.UNDERFLOW;
				if (out.remaining() < 1)
					return CoderResult.OVERFLOW;
				if (!isContinuation(b2 = in.get())) {
					// put second char back by rewinding, putting and re-rewinding
					in.position(in.position()-1);
					in.put(b2);
					in.position(in.position()-1);
					// update last read legal position
					inPos = in.position();
					// append byte directly used
					out.append((char) (b1 & 0xFF));
					// break switch
					break;
				}
				if (!isContinuation(b3 = in.get())) {
					// put third char back by rewinding, putting and re-rewinding
					in.position(in.position()-1);
					in.put(b3);
					in.position(in.position()-1);
					// update last read legal position
					inPos = in.position();
					// append first and second byte directly used
					out.append((char) (b1 & 0xFF));
					out.append((char) (b2 & 0xFF));
					// break switch
					break;
				}
				c = (char) (((b1 & 0x0F) << 12) | ((b2 & 0x3F) << 6) | (b3 & 0x3F));
				// check that we had the shortest encoding
				if (c <= 0x7FF)
					return CoderResult.malformedForLength(3);
				out.put(c);
				inPos = in.position();
				break;
 
			default:
				// parse as latin 1
				out.append((char) (b1 & 0xFF));
				inPos = in.position();
				break;
			}
		}
 
		return CoderResult.UNDERFLOW;
	} finally {
		// In case we did a get(), then encountered an error, reset the
		// position to before the error.  If there was no error, this
		// will benignly reset the position to the value it already has.
		in.position(inPos);
	}
}

Complete source for this can be download here under the regular license.

No related posts.