A better investment then most of the politicans you have probably given to. What do you say?

Waldo, I generally consider you something of a good natured, sharp tongued ass but I must compliment you on Richmond Sunlight. It is an an amazing accomplishment and a model for states everywhere. From reading all of your constant wining I would never have known you had something that big and profound in you.

Great Job. I love it.

*Laugh* Thanks, Halsey…I think. :) I guess you could say I’ve earned my sharp-tongued whining. :)

At work I have an 8-core Mac Pro with 10 gigs of Ram for video editing. I can attest that processing and rendering video is a snap. We ended up ordering the Mac from Apple with 2 gigs and saved tons of money by ordering the remaining 8 from a third party manufacturer.

Oooh, just as I’d hoped. :) Ten gigs—my lord, that’s a lot of memory (hence, good instinct going third party on those). May I some day have the chance to find out for myself how well those eight-cores crunch video. :)

Great Job. I love it.

Good luck.

You mention that EC2 won’t work because (in part) the files involved are too large — have you considered contacting Amazon AWS about this issue? I’m sure they could come up with some very interesting solutions. :-)

Of course, the OCR is the real problem

I’ve never gotten a response from an e-mail to Amazon—I don’t think they’re likely to start now. :) If I could solve the OCR problem (I do that under Parallels in Windows right now, using Abbyy FineReader), then it might actually be worth the upload time—then I could do everything, end-to-end, in EC2. Where I live, I can get a 10MB ADSL, but the price is another $30/month over my existing 1.5MBps connection. But now that I check, I see that only gets me 896kbps upload, which is almost twice as fast as my theoretical upload speed now, but it’s not real hot.

However, I recall that the New York Times used Amazon’s EC2 to OCR all their old pre-electronic archives. I wonder what they used for that?

I wondered the same thing when I read about that. I never did find out. It was Derek Gottfrid who headed that up for the Times—I’ll e-mail him and ask him, if I can manage to track down his address. Thanks for the suggestion!

FWIW, the stuff I’m trying to OCR is pretty straightforward. A screen capture might look like this:

And in this case, there’s no bill number, but there is a legislator name, which I detect (thanks to the shade of blue), hack out, and increase the contrast like such:

As OCRing goes, that’s really not bad. But you can see why it’s important that I use large, high-quality video: the bigger the text, and the less noise around it, the easier it is to OCR it. Googling around this evening, I discovered Cuneiform and Ocrad, two Linux OCR packages that I don’t recall seeing before. I’ll have to give those a whirl.

It’s this process that brings about my two biggest technical obstacles. The first is the size of the video, and the second is the processing time. DreamHost just chokes on the amount of processing that’s required to do all of these transformations, and they just kill the process, no matter how fiercely I renice it. Which is why I need to do all of this on the desktop or, better still, offload it to EC2. EC2 or no, though, I’ll never get around the need to locally rip video, and the quality of video generated by QuickTime (rather than mencoder) is so much higher that I really need to use it to generate the MP4 and the MOV.

Of course, the OCR is the real problem. However, I recall that the New York Times used Amazon’s EC2 to OCR all their old pre-electronic archives. I wonder what they used for that?