Friday, March 03, 2006

"Enigma" message cracked!

This is a story that is on the "front page" of yahoo, so this is not exactly a scoop, but it is really interesting. An amateur cryptographer named Stefan Krah and a whole slew of computers broke the German WWII code created using their "Enigma" code machine. The project started January 9th, and the first code was cracked on February 20.

The project was not "Let's guess a billion times and see what happens." Rather, it was a combination of a huge amount of directed knowledge of how the enigma machine works, how the German language (and German naval terms) works, and how in general cryptography works. Then, using this knowledge, Krah created a computer program that then applies this information to try combinations to crack the code.

The program was written to allow for a coordinated effort of many computers to help break the code. This use of multiple, ganged, computers to share the work on a single task is called "distributed computing." The program runs in the background on a computer and works on these programs when the computer is otherwise idle. Since most people these days use their computer for web browsing or writing letters, computers are basically always idle.

For the first several days (up until Feb 10) only ten computers were working on it. Because of the publicity on slashdot and others, the number of participants was 2,500(!). This is a lot of raw power.

Distributed computing is the "wave of the future" except that it is extremely difficult to do. It is hard to coodinate the actions of more than one computer. As a result, we have a computing model where your computer is sitting idle most of the time asking for web pages on a server that is also serving web pages to a large, large number of other people. Your computer is mostly idle. The server is mostly working. There is a discrepency there. Your computer should share some of the load, but it really doesn't.

I do not have the exact numbers, but I am certain that if you were to add up all the
computing power wasted one hour today, that it would be more than all the computing power that existed in all of 1980. That date could even possibly be moved forward to 1990. There are an awful lot of computers out there doing nothing!

Projects like this show the power of this large pool of untapped computing power. Other projects, such as the SETI project, have done similar work of distributing work across a large number of computers.

The real code to crack is how to crack into this sea of unused computing power! Software has to catch up with the advances in processors and communications -- but it will. The sea of raw computing potential out there is like the seas of oil in the ground in 1870 -- lots of potential, but untapped. It will be tapped, and when it is, wow! You think iTunes and Google are cool, just wait!

Thursday, March 02, 2006

Wikipedia -- Caveat Emptor

There are lots of web reference tools out there, from Onelook to Google to Slashdot and many, many others. Wikipedia is an online encyclopedia with a major twist: Anyone can add to it, and anyone can edit the contents. There is no editorial board. There is no real editing, except that anyone can "edit." Wikipedia even describes itself using its own format -- anyone can edit the entry on Wikipedia!

The theory of Wikipedia is interesting to me, but to my mind the results are extremely uneven. "Editors" with one ax to grind or another modify entries to fit their point of view, which sometime errupt in edit wars. I looked up "Dog" in Wikipedia, thinking it to be a relatively benign subject. Yet, despite the fact that the day is young, there are fourteen edits to the content, some of which were "reverts" back to earlier versions, meaning that someone erased what someone else entered!

To Uncle Mark's mind, a medium in which anyone can contribute with no real fact checking is to be taken with a ton of salt. So, while Wikipedia is good for a quick lookup for obviously true data (like the US is in North America -- no controversy there!), caveat emptor!

Wednesday, March 01, 2006

Yahoo and AOL to Start Charging Senders for Email posted a notice today that aol was going to start charging emailers to email people using their service -- and that "Opposition is growing." Uncle Mark was intrigued by this. What is this?

Well, it turns out that AOL and Yahoo have turned to a company called Goodmail Systems to provide email certification services for them. AOL and Yahoo will accept "certified email" from Goodmail and allow it to go direct to their users' inboxes, bypassing spam filters. Senders who are certified by Goodmail pay a fee for each email sent.

Now -- when I read this, I thought "oh, great! AOL and Yahoo are allowing spammers to spam me, and collecting money for it!"

And, guess what? There is plenty of opposition to this. There is a web site called "Deal AOL." There is the Electronic Freedom Foundation (EFF) take on this. There is's take on this. Why?

The leading claims against this are:

1. This would create a "two tiered" email system where some email is free and some is paid for. The paid for mail would therefore get preferential treatment, leaving small companies and people who do not want to pay for sending email out in the cold.

2. AOL and Yahoo would profit from spam, since they are collecting fees for emails to their user base.

3. It would remove financial incentives for AOL or Yahoo to fight spam -- they would just let spammers "pay to play." [Uncle Mark is adding: If anything, it would incent AOL and Yahoo to force all senders, not just spammers, to use their certification system, and block all non-paying emails].

Goodmail says that they do the following to certify senders:

1. Ensure the company is legitimate and has been in business for a while.
2. Ensure that the senders infrastructure is stable, and has been working for a while (this means that the infrastructure has a history of sending emails).
3. Ensure that the sender is a "responsible emailer" according to their and certified receivers' criteria.

In addition, Goodmail says that people who have email boxes on AOL or Yahoo must "opt in," or agree to receive certified email.

Given this, it became clear that this is not what detracters are saying it is, which is an email "tax." However, it is obviously a fee-based system which does add on a charge per email.

So, what is Uncle Mark's take on this?

I am not against it. I also think that this will die on the vine.

Here's the deal: We need, badly, email authentication. We need to have a system in place that certifies to you that the sender of an emailer is really who he or she says they are. This something that has been on my mind for many, many years, and the problem has gotten worse.

Basically, you have no idea, really, that an email from "Aunt Myrtle" is really from Aunt Myrtle, or is a "spoofed" email. Anyone, and I mean anyone, can make their email program say that they are sending emails from "" This is, frankly, insane, and reflects the openness of the internet, as well as the good intentions of the guys that built it. So we have the following problems:

1. "Phishing" in which people send you emails claiming to be from a bank, or PayPal, or some other service, with the intention of stealing your money or identity. This is a real problem.
2. Viruses that send copies of itself while claiming to be from someone else.
3. Internet scammers can claim to be anyone they want to be.

And there are others. People spend a ton of money on filtering "spam" and viruses from emails. Why? Because we have no real idea where the email came from. Spam filters and services can do quite a bit to filter this out -- you are spending money to do this, it is NOT free.

Let's say we have this world: I am Mark Patterson, Uncle Mark. I have certified to the government or to some authentication agency that I am really Mark Patterson, and they checked my ID, and they confirm that I am indeed Mark Patterson. They issue me an "email passport" that says that email sent from me really came from me, and no one else. Now let's say that everyone does this: Everyone applies for an "Email Passport."

Now, let's say that someone with an email passport emails me. I now know who they are, because I see they have a passport, and I trust the agency that issued it. So, Aunt Myrtle emails me, and I can see that it really is her.

What does that do?

1. Phisher's can no longer claim to be from a bank, because they do not have the bank's passport.
2. Spammer's can no longer spam with impugnity, because you can block all emails with their passport.
3. Viruses are limited in their effectiveness. You can still get a virus sent, but you will know where it came from, unlike now.

Basically, you now who is sending you email. If an email comes in that does not have a passport, that email is immediately suspect. You will know, however, that it is not a bank if is claiming to be one, because it does not have the bank's passport.

So why not just have an email passport?

Well, the main reason is that there are conflicting standards for email authentication, and there is no consensus. Microsoft is pusing a technology called "Sender ID", and Yahoo and others are pushing "Domain Keys," and then there is the "Sender Policy Framework." That's just three. This is something that needs to be a single standard, like email in general is.

So, it looks like AOL and Yahoo are cutting the Gordian Knot and saying "Heck with it! You gotta pay to reach our users!"

So, why will this "pay to play" scheme die on the vine?

1. The end user must "opt in" to receive it. Would you opt in to get paid-for advertising?
2. The sender must pay to send each email. Would you, as a company, pay to send anything but revenue-generating advertisements?
3. If Yahoo or AOL play the hand to heavily, and force people to get these emails, people will use other email services, or filter out these emails on their own. This removes the economic incentive of the senders.
4. If Yahoo or AOL neglect their spam and virus filters, users will leave.
5. Banks and other companies that want to ensure their emails are certified will certain use other means than a "pay per email" service. They will adopt an email authentication service, or all of them.

In addition to the above, that larger issue is that Internet services "want to be free." What that means is that once you have a connection to the internet, it is really, really easy to communicate with other people on the internet, and all forms of this communication therefore can be cheap to build and use. AOL's IM and Yahoo Messenger are free. If they started charging real money for it, something else would come along and do the same thing cheaper. Email is really, really cheap. If someone starts charging money for it, it is really, really cheap to go somewhere else. Voice over IP and internet phone calls are really, really cheap. If someone starts to charge big cash to make an internet phone call, you can go somewhere else -- even use IM or Yahoo Messenger. [This is why Uncle Mark was so flabbergasted when eBay spent so much money for internet calling vendor "Skype." You can do this for nothing! If you know even a bit about programming, you can write your own internet phone!]

So -- mark me -- this will start off, and then in about a year, it will be a non-issue. AOL and Yahoo may still use it, but my guess is that it will be very low volume. It certainly will not replace email, or, unfortunately, get rid of spam.