All posts by Waldo Jaquith

Waldo Jaquith (JAKE-with) is an open government technologist who lives near Char­lottes­­ville, VA, USA. more »

“Accidental APIs”: Naming a design pattern.

Like many open data developers, I’m sick of scraping. Writing yet another script to extract data from thousands of pages of HTML is exhausting, made worse by the sneaking sense that I’m enabling the continuation of terrible information-sharing practices by government. Luckily, it’s becoming more common for government websites to create a sort of an accidental API—populating web pages with JSON retrieved asynchronously. Because these are simply APIs, albeit without documentation, this is a far better method of obtaining data than via scraping. There is no standard term to describe this. I’ve been using the phrase “accidental API,” but that’s wrong, because it implies a lack of intent that can’t be inferred. (Perhaps the developer intended to create an API?)

Recently, I solicited suggestions for a better name for these. Here are some of my favorites:

The best ones are immediately understandable and don’t ascribe intent on the part of the developer. I suspect I’m going to find myself using Bill Hunt’s “incidental API” and my (and Tony Becker’s) “undocumented API.” I particularly like “undocumented API” because it begins with the assumption of competency on the part of the developer, and that the only shortcoming of the API is its documentation, but I’ll try out a few of them in the coming weeks and see what sticks.

Dynamic electrical pricing demands dynamic price data.

The power industry has begun its long-anticipated shift towards demand-based pricing of electricity. Dominion Power, my electric company here in Virginia, has two basic rates: winter and summer. Although the math is a bit complicated, electricity costs about 50% more in the summer than in the winter, averaging 12¢ per kilowatt hour. (One can also pay for sustainably sourced energy, as I do, and this raises these rates by 1.3¢ per kilowatt hour.) While this price system is very simple, it is also bad, because it fails to respond to consumer demand or the realities of electrical generation.

Here’s an explanation of the problem and the proposed solution: open electrical rate data.

Excess Demand

On a very hot day—say, north of 100°F—everybody wants to keep their house at 72°. This requires a great deal of electricity, which means that Dominion has to generate a great deal of electricity. And that’s fine, because people are paying per kilowatt hour. If they want to pay $1 an hour to keep their house cool, that’s their prerogative. They pay, and Dominion uses the money to run their plants. But this all starts to fall apart when Dominion nears its maximum capacity.

As demand approaches capacity, Dominion is faced with a dilemma. Like most power companies, Dominion probably has a standby boiler in their coal-based power plants. This is not normally fired up, because it’s the oldest, polluting-ist boiler that they have. This boiler falls well below the modern standards of efficiency within state and federal regulations. Turning it on might increase by tenfold the power plant’s emissions of regulated pollutants, and guarantees that they’re going to be paying fines. At 10¢ per kilowatt hour, running their modern boilers is a profitable enterprise, but running the ancient, standby one is a money-losing endeavor.

In order to avoid brown-outs—demand exceeding capacity, resulting in insufficient amounts of power being delivered to customers—Dominion has to start up this nasty old boiler, even though they might only be needed to provide power to a few thousand customers. The incremental cost of serving these few customers is enormous, but necessary to keep the whole enterprise going.

Worse still, imagine if the temperature continues to climb. Demand spikes further. More power is needed than Dominion can generate or buy from other power companies (who are dealing with the same problem). Brown-outs or rolling blackouts are now impossible to avoid. Customers are angry. Dominion is losing money.

Dynamic Pricing Models

Enter dynamic—aka “demand-based”—pricing. There are two ways that dynamic pricing can work.

Dominion's summer rate plan.
Dominion’s summer rate plan.

The first dynamic pricing model is based on a schedule of rates relative to demand. This tells customers how much power costs on low-demand days versus high-demand days, with any number of gradients between the two. And within that daily rate difference, there are price changes throughout the day. A low-demand day might average around 9¢ per kilowatt hour, and a high-demand day might top out at 20, 30, even 50¢ per kilowatt hour. The advantage of this system is that it’s controlled and limited—people know what the possibilities are, and there’s a theoretical cap on how much power can cost. The disadvantage to this system is that there’s no way for customers to know how much collective demand exists. While Dominion understands that a high-capacity day is anything north of (say) 25,000 megawatts, customers have no way of knowing how high that collective demand is. This is an actual system that exists around the nation right now, and that Dominion allows customers to opt into.

The second dynamic pricing model is based on a real-time auction of electrical rates. For this approach to work, you’d tell your appliances how much you’re willing to pay to run them. You’ll pay no more than 35¢ to dry a load of laundry. You’ll pay no more than $2.50/day to keep your house cool, unless your house gets above 78°, in which case you’ll pay up to $5.00/day. Your water heater will keep water at 130°, unless power goes above 15¢ per kilowatt hour, in which case it will drop to 120°. And so on. Then your home power meter aggregates this data, and makes bids for power, bidding against every other customer. This works somewhat like eBay’s automatic bid system, and very much like Google Ads’ pricing model. Of course, this infrastructure does not exist yet, and so this is entirely in the realm of the imaginary. Still, I feel comfortable saying that this system is inevitable.

Returning to the reality of the first model—a published rate schedule—there’s a serious problem with information asymmetry. How is one to know the cost of electricity at any given time, if you don’t know if it’s a low-, medium-, or high-cost day? Dominion’s solution to this is both straightforward and complicated: they’ll e-mail you at 6 PM every day and tell you which of three rate structures that they’ll use the following day. Each rate structure changes over the course of the day, with different prices overnight, in the morning, through the bulk of the day, and in the evening.

But, wait, it gets harder. Dominion also institutes a “demand charge.” Every half hour, they sample how much power that you’re using at that moment. Then your monthly bill has a fee based on the largest amount of power that home was using at one of those sampled moments in the prior 30 days. If you used no power all month, except for one minute in which you used a very large amount of power, you would be billed a corresponding large amount, despite your near-zero average.

For customers, Dominion’s approach is dizzying. It requires that people keep track of electrical rates on a day-to-day and hour-to-hour basis, peak home power usage at all times, and provides nothing that would support the growing industry of home automation and energy saving devices, which could manage electrical use automatically. The popular Nest thermostat can be automatically reprogrammed via the internet. Apple recently announced an entire platform of home automation tools, controllable and configurable via iPhone, iPad, or desktop computer. Philips makes a light bulb kit that permits each bulb to be controlled remotely, the brightness and color of the bulbs configurable individually. There’s a whole ecosystem of hardware, software, and data to allow one’s home’s energy use to be adjusted in response to external factors. But what they can’t do is read Dominion’s e-mails at 6 PM every night. That’s an unbridgeable air gap, a failure on the part of Dominion that is perhaps mystifying or perhaps rational, depending on one’s level of cynicism.

Open Electrical Rate Data

There’s a simple solution to this: open electrical rate data. In addition to sending out an e-mail at 6 PM every day, Dominion could maintain a file on their server that provides machine-readable data about current and near-future power rates. It might look like this:

Right now, the closest that they get is a retrospective page, which has allotted space for the next day’s price (“Classification for tomorrow:”), but the page text ends with the colon—I’m yet to see that classification provided. [Hours after I published this, Dominion finally wrote something in that space, I assume prompted by the 90°F forecast.]

If this data was provided, it would be trivial to use it to enable home automation and energy management tools to schedule and control energy-intensive home services and appliances.

And, in fact, that’s a feature that Nest supports. The thermostat will dynamically adjust the temperature of a home during the highest-priced periods, generally very hot summer afternoons and very cold winter nights. But because precious few power companies provide the necessary data to support this feature, it’s not useful to most Nest customers. Nest doesn’t provide a comprehensive list of participating power companies, and after searching through their press releases and trying out a handful of ZIP codes from across the country in their form, I have to conclude it’s because there are very few participating power companies.

Publishing open electrical rate data is not difficult. If they can send out an e-mail, they can certainly update a JSON file. For a competent developer, it would be an afternoon project. A company that is capable of managing an entire electrical grid and the entire usage tracking and billing system that accompanies it is certainly capable of a tiny project like this.

I’ll warrant that Nest—which is owned by Google—is in a good position to establish a standard JSON schema for power companies to use. Some power companies would probably welcome being told what schema to use, giving them one fewer thing to worry about. Right now, it appears that Nest is basically taking any data that they can get. (It wouldn’t shock me to find out what they’re intercepting night-before e-mail alerts and using those to update thermostats with rate data.) Power companies are going to catch on to the enormous importance of rate data, and Nest has the first-mover advantage. I hope that Nest puts together an uncomplicated schema, advertises it on a developer page, encourages existing and new partners to publish in that schema, and eventually requires that participating power companies comply with their schema, assuming that they end up in a position where they can make such demands.

Open electrical rate data will provide real savings to consumers and utilities alike. It’s a necessary and inevitable development in power distribution and home automation. I hope that power companies and Nest take the simple steps necessary to usher in this era of open energy data, and soon.

What’s wrong with Puckett’s resignation?

Further to the matter of Sen. Phil Puckett’s retirement, I want to play out the shades of inappropriateness here. While what he has done clearly feels wrong (allegedly quitting his seat in the Senate of Virginia in exchange for a job running a state-chartered organization and a judgeship for his daughter, all done immediately prior to the deadline for the legislature to hold a vote on the budget, in which his absence will give Republicans a one-member majority and the ability to prevent healthcare reform), I think it’s worth exploring what about it is wrong.

Imagine that Sen. Puckett had sold his vote. In exchange for $150,000 in cash, he would vote against a budget that included healthcare reform. Any reasonable person would regard that as wrong.

Imagine that Sen. Puckett had sold his non-vote. In exchange for $150,000 in cash, he would take a walk when the bill came up for a vote. I think that any reasonable person would regard that as wrong, too.

Imagine that Sen. Puckett had sold his absence. In exchange for $150,000 in cash, he would make sure that he was thousands of miles away when the legislature reconvened to hold the vote. I also think that any reasonable person would regard that as wrong.

Now imagine that Sen. Puckett sold his resignation. In exchange for $150,000, he would quit so that he could not cast a vote on the budget bill. Many reasonable people would regard that as wrong.

And now we have the alleged reality, of Sen. Puckett selling his resignation in exchange for perhaps $150,000 annually, so that he could not cast a vote on a budget bill. Many reasonable people would also regard that as wrong.

In that real-life scenario, we have two possible parties who might have done something wrong. First, we have Sen. Puckett, who may or may not have intended to quit in order to prevent a vote from happening—he might argue that he simply quit the legislature for a tantalizing job offer that was only available within a small window, but that he couldn’t have held while also serving in the legislature. Puckett of course knew that his resignation would prevent him from voting on the most important bill before the legislature, and unless he is a very stupid man, he would have known that Kilgore was offering him the job so that he could not cast that vote. Then we have Del. Terry Kilgore, the chair of the tobacco commission, who offered Puckett the job. Kilgore likewise knew what Puckett’s resignation would mean. I suspect that two key questions here these: Did Kilgore intend to prevent Puckett from voting on the budget bill by offering him a job? And did Puckett intend to not vote on the budget bill in exchange for accepting a job?

Kilgore, of course, thinks that he’s being clever by saying that he never offered Puckett a job, but that “if he’s available, we would like to have him.” This is almost certainly bullshit, which I define as meaning that the statement is a) untrue, b) Kilgore knows that it’s untrue, and c) we know it to be untrue. It insults our collective intelligence to claim that Puckett resigned from the Senate of Virginia on the vague hope of a job heading the tobacco commission. (Or, at least, it insults Puckett.) And that leads us to the third key question: Have Puckett and Kilgore already negotiated the terms of employment for the tobacco commission?

If state investigators look into a violation of § 18.2-447, they’re going to get records of communications between Puckett and Kilgore. If they actually struck a deal here, and they didn’t have the good sense to only negotiate the terms privately and in-person (making discovery impossible), this could get ugly, and fast. On the other hand, if they’re aboveboard and have any sense, they made sure that all negotiations occurred in the presence of attorneys, and were done either exclusively by writing or were recorded as audio or video.

Evidence has a way of disappearing. I hope investigators are looking into this.

Ethics and tobacco meet again in Richmond.

This evening, the Washington Post’s Laura Vozella covered big political news:

Republicans appear to have outmaneuvered Gov. Terry McAuliffe in a state budget standoff by persuading a Democratic senator to resign his seat, at least temporarily giving the GOP control of the chamber and possibly dooming the governor’s push to expand Medicaid under the Affordable Care Act.

Sen. Phillip P. Puckett (D-Russell) will announce his resignation Monday, effective immediately, paving the way to appoint his daughter to a judgeship and Puckett to the job of deputy director of the state tobacco commission, three people familiar with the plan said Sunday.

This is, it must be said, a brilliant bit of maneuvering on the part of Republicans in the legislature. With the Senate split 20-20 between parties (and the Democratic lieutenant governor able to act as the tiebreaker on only certain matters), persuading Puckett to quit is a clever move. No doubt his Republican colleagues were aware that he was interested in retiring after 17 years in office. (He’s up for another 4-year term in November of next year, so it’s decision-making time for him.) By offering him a plum job, contingent on quitting his job before the long-delayed budget vote could be held, they will regain a majority in both chambers of the General Assembly, making the governor’s demand unreasonable. Politically speaking, well-played.

That said, this is bullshit. In a legislative session that had a major theme of ethics reform, during which Governor Bob McDonnell was indicted (the week after his term ended, an apparent Justice Department courtesy), wrapping it up with a political coup contingent on trading control of an entire chamber for a state job for a legislator and a judgeship for his daughter? It’s absurd. It’s the stuff of parody. This would be the kicker at the end of an episode of Alpha House.

I don’t think it’s tone-deaf on the part of Republicans, I just think that they don’t care. And Puckett’s constituents almost certainly say that they are opposed to “Obamacare,” when asked in a survey, and his new job—handing out hundreds of millions of dollars to rural areas—will make him very popular in his district. Basically, he gets the job of Santa Claus. So not only will his (former) constituents probably not object, but he’s out of office anyway, so what they think also doesn’t really matter.

Is Puckett’s daughter qualified to be a judge? Do Republicans care? Does Puckett care? Is this how we want to be selecting our judges? Is it appropriate for a legislator to require that his own daughter be made a judge? Shouldn’t, in fact, the legislator have absolutely nothing to do, in any way at all, with his daughter’s potential appointment to a judgeship?

The worst part is that this isn’t illegal. State ethics laws might as well not exist. We’ve only had a couple of state-level elected officials indicted on ethics charges in recent history.

(At Think Progress, Ian Millhiser argues that this could actually be illegal, citing § 18.2-447(2). I hope that’s the case.)

First was Delegate Phil Hamilton, who in 2009 was caught holding a state job that he created legislatively, in exchange for being awarded the job. Especially problematic was that he apparently never even showed up for the $40,000/year part-time job. He’s in prison now, and will be through the rest of the decade.

Second was Gov. McDonnell, indicted in January after accepting hundreds of thousands of dollars in gifts from a major campaign donor in exchange for favorable treatment of and promotion of that donor’s sham medical treatments. Declared McDonnell, of Star Scientific’s flagship product, unregulated supplements: “They work for me!” (Just this week they changed their name to Rock Creek Pharmaceuticals, in what will be a fruitless effort to outrun their terrible reputation.) The indictment charges McDonnell with honest-services fraud, false statements, conspiracy, and obstruction of justice, among other things.

Let us now consider a series of odd coincidences here. Puckett was wooed away to run the Tobacco Indemnification and Community Revitalization Commission, which paid $309M to Virginia tobacco producers to persuade them to stop growing tobacco. The organization is funded by the Tobacco Master Settlement Agreement, which was the result of a lawsuit that 46 states brought against Altria (née Philip Morris), R.J. Reynolds, Brown & Williamson and Lorillard. In 1998, the companies settled, agreeing to pay $206 billion to those states. Virginia’s share is distributed via the state-chartered organization that Puckett will be headed up. And then there’s the company that brought McDonnell down, Rock Creek Pharmaceuticals, née Star Scientific, née, Star Tobacco. Star Tobacco manufactured cigarettes and chewing tobacco, and they remained in the tobacco business straight through 2012, as they sought to rebrand themselves as a pharmaceutical company. (Pharmaceuticals—or, rather, “supplements”—manufactured with tobacco, appallingly.) Although Star Tobacco refused to join the settlement, states ultimately got their money via Brown & Williamson, which manufactured Star Tobacco’s products in their plants. So we’ve got Puckett going to work for the organization that was funded by the profits generated by the company that allegedly bribed McDonnell.

But, wait, there’s more. The tobacco commission actually gave a $2M grant to Star Tobacco to help them promote their cigarettes and chew, appallingly. And—get this—a big part of what McDonnell is in trouble for was that he allegedly tried to persuade the tobacco commission to provide a grant to Star Scientific, by way of funding clinical trials of one of their tobacco supplements at the University of Virginia and Virginia Commonwealth University, to provide a sheen of legitimacy to what is now clearly a useless substance.

And it gets better still! The chairman of the tobacco commission is Terry Kilgore, the Republican legislator who is said to have struck the deal with Puckett. Kilgore’s twin brother, Jerry Kilgore, isn’t merely a former attorney general of Virginia, but he’s also the attorney representing Johnny Williams, the CEO of Star Scientific, in the McDonnell case.

Here’s the thing that you should know, if you’re not from Virginia: tobacco isn’t a big business here. It hasn’t been for a long, long time, and the tiny shreds of it that were left fled the country with the passage of NAFTA. While it’s true that the General Assembly has tobacco leaves painted on the ceiling of Capitol Rotunda, but that’s a holdover from decades ago, long before Puckett or McDonnell were in office.

These two tobacco-related scandals aren’t about the power of tobacco, it’s about the profound lack of ethics laws in Virginia.

We talk a good game here about “the Virginia Way.” This is the notion that rhetorical civility, bipartisanship, comity, and transparency are all that’s really necessary. We don’t need laws about ethics, because everybody in government is honest and everybody in the public knows it. We don’t need regulation of businesses, because their inspections are on file in a cabinet in Richmond where anybody can go between the hours of 9–5 (but not during the lunch hour) and ask to look at them, and isn’t that transparency? The people in power claim to truly believe this, and they put on a very convincing face while explaining that they truly believe it. I know these people. I have spoken on panels with these people (in opposition to them, confessedly), and I have put on events at which they have spoken about the Virginia Way. It is now entirely evident that these people are utterly, humiliatingly wrong, but, much like legislative Republicans and Phil Puckett right now, they just don’t care, because it’s working out for them.

Kilgore and company will get their majority, and they’ll be able to keep 400,000 Virginians from getting health insurance and the state from getting $1.7B in federal taxes already being collected from Virginians, which are very important to them for some reason that they can’t quite explain. Puckett will get a job, with a salary that Kilgore says he’ll determine at a later date. (Can you believe that Puckett is quitting his Senate seat in exchange for a job with an undetermined salary? Yeah, neither can I.) Apparently that moral calculus—he gets a job, 400,000 people don’t get health care—makes sense to him.

This entire sad show—bound to make Virginia a national laughingstock yet again—is the cherry on top of the General Assembly’s completely useless ethics reform bill which, of course, does absolutely nothing to prohibit this. Selling your seat in exchange for a couple of state jobs is something that they never considered, despite its obviousness, because the entire bill was a charade, a performance for the benefit of voters. Well, perhaps “benefit” isn’t the word. The only people benefiting from this are the 140 members of the legislature. I can think of nineteen senators who might be wishing they’d written a better bill.

Opening up Virginia corporate data.

In Virginia, you can’t just get a list of all of the registered corporations. That’s not a thing. If you dig for a while on the State Corporation Commission’s website, you’ll find their “Business Entity Search,” where you can search for a business by name. But if you want to get a list of all businesses in your county, all businesses that have been formed in the past month, all businesses located at a particular address, etc., then you’re just out of luck.

Except. The SCC will sell you their database of all 1,126,069 companies. It’s not cheap, at $150/month, with a minimum three-month commitment. You have to sign a five-page contract, and the data is a hot mess, of no value to anybody other than a programmer.

So, naturally, I wrote the SCC a check for $450 at the end of April, bought the data, and now give it away for free. (Updated weekly, early Wednesday morning, I automatically transfer the enormous file to https://s3.amazonaws.com/virginia-business/current.zip.) Because it’s not right that people should have to pay for public data. The SCC is already generating this data, and they’re already hosting the file on their website—why sell it? We’ve already paid for it, out of our taxes and out of our business incorporation fees. I FOIAed the list of customers for this data. There are just six, so it’s not like this is a money-making endeavor for the SCC. (Only one of them, Attentive Law Group, is in Virginia.)

Now people can have this terrible file, useful only to programmers. So what are they to do with that file? Well, maybe nothing. So I’ve also written some software to turn that data into modern, useful formats. Named “Crump” (for Beverley T. Crump, the first-ever member of the State Corporation Commission), it is, naturally, free and open source. Crump turns the SCC’s fixed-width text file into JSON and CSV files. Optionally, it will clean up the data and produce Elasticsearch import files, basically allowing the data to be quickly loaded into a database and made searchable. Again, anybody can have the data for free, and anybody can have Crump for free, to turn that data into useful data.

And, finally, I’ve created a website, creatively named “Virginia Businesses,” where non-programmers can access that data and do things with it. I’ve barely gotten started on the website—at this point, one can download individual data files as either CSV or JSON, download the original data file from the SCC, or search through the data. The search results are terrible looking, and not all of the data is loaded in at the moment, but by the time you read this blog entry, perhaps that will all be much improved. I intend to add functionality to generate statistics, maps, charts, etc., to let people dig into this really interesting data. The website updates its data, automatically, every week. Naturally, the website itself is also an open source project—anybody can have the website, too, and can set up a duplicate to compete with me, or perhaps create a similar site for another state.

So, free data, free software, and a free website. There’s no catch.

OpenCorporates, whose excellent work inspired this project, has imported the data into their own system, meaning that Virginia’s corporate data is now available in a common system with 69 million other corporations from around the U.S. and the world.

Then, a couple of weeks ago, a happy surprise: the Shuttleworth Foundation e-mailed me, out of the blue, informing me that they’re giving me $5,000 to support my work in open data, as a part of their “flash grant” program. I can do whatever I want with that money, and I’m going to use a chunk of it to support this work. That means that I’m not out of pocket on that $450 check, and that I can continue to pay for this data for a while, so that others can continue to benefit from it.

I don’t know where this project is going—it’s just a hobby—but even if I stopped doing any more work on it tomorrow, I know I’d be leaving Virginians with much better business data than they had before.

In addition to the Shuttleworth Foundation, my thanks to the ACLU of Virginia and the EFF for providing me with legal advice, without which I couldn’t have even begun this project, and to Blue Ridge InternetWorks, who generously donates the website hosting and server power to crunch and distribute all of this data.

Cloud corporations.

Many months ago, my friend Tim Hwang told me that he’d like to see an API created for corporate registrations, because that would enable all kinds of interesting things. Tim runs the semi-serious Robot Robot & Hwang, a legal startup that aspires to be a law firm run entirely in software. I’ve been chewing over this idea for the past year or so, and I’m convinced that, writ large, this could constitute a major rethinking of the Virginia State Corporation Commission. Or, really, any state’s business regulation agency, but my familiarity and interest lies with Virginia. But first I have to explain Amazon Web Services. (If you don’t need that explained, you can skip past that bit.)

Amazon Web Services

Not so long ago, if you wanted to have a web server, you needed to actually acquire a computer, or pay a website host to do so on your behalf. That might cost a couple of thousand dollars, and it took days or weeks. Then you had to set it up, which probably meant somebody installing Linux or Windows from CD-ROMs, configuring it to have the software that you needed, mounting it in a rack, and connecting it to the internet. You’d have to sign a contract with the host, agreeing to pay a certain amount of money over a year or more in exchange for them housing your server and providing it with a connection to the internet. That server required maintenance throughout its life, some of which could be done online, but occasionally somebody had to go in to reboot it or swap out a broken part. But what if your website suddenly got popular, if your planned 100 orders per day turned into 10,000 orders per day? Well, you had to place orders for new servers, install operating systems on them, mount them in more racks, and connect them to the internet. That might take a few weeks, in which time you could have missed out on hundreds of thousands of orders. And when your orders drop back to 100 per day, you’ve still got the infrastructure—and the bills—for a much more popular website.

And then, in 2006, Amazon.com launched Amazon Web Services, a revolutionary computing-on-demand service. AWS upended all of this business of requisitioning servers. AWS consists of vast warehouses of servers that, clustered together, host virtual servers—simulated computers that exist in software. To set up a web server via AWS, you need only to complete a form, select how powerful of a server that you want, agree to pay a particular hourly rate for that server (ranging from a few cents to a few dollars per hour), and it’s ready within a few minutes. Did your planned 100 orders turn into 10,000? No problem—just step up to a more powerful server, or add a few more small servers. Did your 10,000 orders go back to 100? Scale your servers back down again. Better still, AWS has a powerful API (application programming interface), so you don’t even have to even intervene—you can set your own servers to create and destroy themselves, control them all from an iPhone app, or let software on your desktop start up and shut down servers without any involvement on your part.

There are other companies providing similar cloud computing services—Rackspace, Google, and Microsoft, among others—but Amazon dominates the industry, in part because they were first, and in part because they have the most robust, mature platform. There remain many traditional website hosts, which you can pay to house your physical servers, but they’re surely just a few years away from being niche players. Amazon did it first, Amazon did it best, and Amazon is the hosting company to beat now.

Cloud Corporations

Imagine Virginia’s State Corporation Commission (SCC) using the Amazon Web Services model. Virginia Business Services, if you will. One could create a business trivially, use it for whatever its purpose is, and then shut it down again. That might span an hour, a day, or a week. Or one could start a dozen or a hundred businesses, for different amounts of time, with some businesses owned by other businesses.

Why would you do this? This is actually done already, albeit awkwardly. Famously, the Koch brothers maintain a complicated, sophisticated web of LLCs, which they create, destroy, and rename to make it difficult to track their political contributions. This probably costs them millions of dollars in attorneys’ fees alone. Doing so is perfectly legal. Why should that only be available to billionaires? Or perhaps you want to give a political contribution to a candidate, but not in your own name. Wealthy people create a quick LLC to do this. Maybe you want to host a one-off event, or print and sell a few hundred T-shirts as a one-time thing—a corporate shield would be helpful, but hardly worth the time and effort, except for the wealthy. There’s no reason why the rest of us shouldn’t be able to enjoy these same protections and abilities.

Cloud corporations would be particularly useful to law firms who specialize in managing legal entities. Right now, they spend a lot of time filing paperwork. Imagine if they could just have a desktop program, allowing them to establish a corporation in a few minutes. Instead of charging clients $1,500, they could charge $500, and make an even larger profit. Although surely Delaware would remain attractive for registering many corporations, due to their friendly tax laws, the ease of registering a corporation in Virginia would surely make it attractive for certain types of business.

So what would the SCC need to do to make this happen? Well, right now, one can register for an account on their site, complete a form on their website, pay $75 via credit card, and have a corporation formed instantly. From there on out, it costs $100/year, plus they require that an annual report be filed. Both of these things can be done via forms on their website. (Note that these dollar values are for stock corporations. There are different rates for non-stock corporations and limited liability corporations.) All of which is to say that they’ve got the infrastructure in place for purely digital transactions.

But to support to an AWS model, they’d need to make a few changes. First they’d have to expose the API behind those forms, to allow programmatic access to the SCC’s services. Then they’d have to add a few new services, such as the ability to destroy a business. And they’d need to change their pricing, so that instead of being billed annually, pricing would be based on units of weeks, days, or even hours. (That pricing could be elevated significantly over standard pricing, as a trade-off for convenience.) The SCC has some antiquated regulations that would need to be fixed, such as their requirement that a business have a physical address where its official documents are stored (“Google Docs” is not an acceptable location). Finally, to do this right, I suspect that the Virginia Department of Taxation would need to get involved, to allow automated payment of business taxes (something that Intuit has spent a great deal of money to prevent) via an API.

Next Steps

I regret that this is unlikely to happen in Virginia. The State Corporation Commission is like its own mini-government within Virginia, with its own executive, legislative, and judicial functions, and seems accountable to nobody but themselves. FOIA doesn’t even apply to them. They’re not known as a forward-thinking or responsive organization, and I’m dubious that either the legislature or the governor could persuade them or even make them do this.

But I am confident that some state will do this (I hope it won’t be Delaware) and that, eventually, all states will do this. It’s inevitable. Whoever does it first, though, will enjoy a first-mover advantage, perhaps on the scale of Amazon Web Services. I’ll enjoy watching it. Maybe I’ll even register a few corporations myself.

Modern tools of business.

The last time I started a business was over a decade ago. Good news: it’s gotten a lot easier. There was a lot of overhead in startup costs that have been reduced to little or nothing, due to the rise of internet-based tools. I had to do some research and try a lot of tools before I settled on the basic suite. Here’s what I’m using at the U.S. Open Data Institute:

CRM

Contactually watches my e-mail and serves as a repository for notes after meetings and phone calls. I associate each contact with a given project and classify them based on the nature of our relationship (client, funder, employee, vendor, colleague, board member, etc.) Based on those criteria, it reminds me when I need to get in touch with somebody. $20/month.

Task Management

The Asana group task management software is almost like using native Mac OS X software. Everything I need to do goes in here, attached to a given project and categorized. As the US ODI adds more employees and contractors, the group functionality will start to pay off. Free for teams of up to 15 people.

Assistant

I’d included an assistant in my budget, and the cost was non-trivial. In reality, the amount of assistance that I’ll need is going to fluctuate over time, and at this point my need is so minimal that I don’t even know how to meaningfully employ a competent person for, say, 4 hours a week. Then a friend suggested Zirtual. Now I have an assistant in North Carolina, who I can call or e-mail when I need some help. (Admittedly, I’m still figuring out how to work with an assistant, accustomed as I am to doing everything myself, but that’s not Zirtual’s fault.) $200/month for 8 hours of work.

Teleconferencing

Most teleconferencing systems are terrible, and I’ve generally relied on the folks on the other end of phone line to arrange something. Now I use UberConference. It has a simple web-based interface, allows selective muting of participants’ phones (that one guy with the barking dog), it doesn’t require a PIN, and it’ll even call the participants when the conference starts, rather than vice-versa. Free, $10/month for a bunch of nice features, or $20/month for those nice features and a toll-free number.

Phone/PBX

There was no way I was going to pay for a landline. And I’ve already got a mobile phone, of course, so I didn’t need a second one of those. But I needed an organizational phone number, voicemail, an employee phone directory, etc. That’s where Grasshopper comes in. They host the PBX, and connect calls to my phone. Voicemails are e-mailed to me, or available via an iPhone app. Starts at $12/month.

E-Mail

Like most everybody, I host organizational e-mail via Google Apps. Not only is it drop-dead simple, but their support of two-factor authentication gives me one less thing to worry about. $5/month/user.

Office Suite

The days of having to shell out a few hundred bucks for Microsoft Office are over thanks to, again, Google Apps. $5/month/user.

Website Host

The US ODI’s website is hosted on GitHub Pages. Cost: $0. I’ve got an Amazon Web Services account should I need to host anything that won’t work there, which I can just drop some files into S3. Cost: literal pennies.

The grand total is $279/month, with $200 of that for Zirtual. The equivalent services a decade ago would have been a great deal more expensive, or more crude. I’m not sure what’s been more powerfully beneficial to entrepreneurship: the shift towards internet-hosted business services, or the Affordable Care Act. Perhaps we’ll know in a few years.

Announcing the U.S. Open Data Institute.

I’ve joined a new endeavor this week—the U.S. Open Data Institute. Today is just day #2 for me, and for the organization. The US ODI is modeled on the UK-based Open Data Institute, a year-old organization that’s bridging the gaps between government and the private sector. That’s what we intend to do at the US ODI—help government, businesses, non-profits, and individuals make more effective use of the data being produced by governments and, in some cases, businesses. That’ll be done largely through facilitating collaboration between existing organizations and government agencies, and also by working one-on-one with government agencies who need help opening up their data. Neither is particularly glamorous—basically playing matchmaker and running a free IT help desk—but it’s what needs to be done to unlock the annual $3 trillion in economic value that’s waiting to be capitalized on. A whirlwind of activity has surrounded the establishment of this organization, to which I’ve largely been a stunned witness, with particular credit going to the Knight Foundation, the Aspen Institute, the White House Office of Science and Technology Policy, the Open Data Institute, Daniel X. O’Neil, and Max Ogden, although dozens of other people and organizations played important roles.

For  a lot more detail, see the Knight Foundation’s announcement of their $250,000 in funding, my blog entry on their site about the US ODI, the White House’s blog entry, US ODI board chair Daniel X. O’Neil’s blog entry, or Robinson Meyer’s especially fun and detailed Atlantic article.

“I put Algernon’s body in a cheese box and buried him in the backyard. I cried.”

I’m a smart guy.

I’m not bragging. It’s not like I deserve any credit for being smart. I didn’t do anything to make that happen. I’m also tall, but nobody compliments me on that. (Good job being tall!) Intelligence is an immutable characteristic that I have at times capitalized on and at other times let go to waste. So it goes.

It’s not easy to understand what it’s like to be smarter or dumber than one is. I have some friends who are much smarter than me, and I can’t pretend to know what it’s like to experience the world as they do. Like height, intelligence is a privilege, in that we live in a society that places value on both. And although I don’t know what it’s like to be of a different intelligence, I do know what it’s like to be short; we all started out short. Although I have brown hair now, I bleached my hair a couple of times as a teenager, so I know what it’s like to be blond. If I really wanted to, I could probably fake my way into at least being perceived as of a different race, sex, or sexual orientation.

Interestingly, I have come to find out what it’s like to be dumb. I am, at this moment, rather stupid.

On September 1 I returned from a trip to Buenos Aires. During the 14 hours that I spent waiting around the Atlanta and Charlotte airports that day, I became progressively more sick, until I had a full-blown case of influenza. (It’s flu season in the southern hemisphere, after all.) Ten days later, I was still sick. The fever was gone, and my symptoms were muted, but I still felt sick. I didn’t want to get out of bed, I didn’t care about eating, I just wanted to lie around and watch TV.

So I went to the doctor, starting to doubt my self-diagnosis. After explaining my symptoms, both physical and mental, he came to a conclusion quickly: I have Lyme disease. (A pending blood test may help to confirm that, but they’re famously unreliable.) I was bitten by a tick this spring—one of many ticks that I find embedded in my skin each spring, summer, and fall—that was bearing Lyme spirochetes, which it injected into me. The immune-suppressing tick saliva allowed  the bacteria to establish an infection there. That initial infection raised a nickel-sized welt on my back, which I presented to my dermatologist, who assured me it was nothing to worry about. (In fact, this may well have been a borrelial lymphocytoma.) Gradually, those spirochetes reproduced, spreading throughout my body, through my bloodstream. Some of those spirochetes have hijacked my own cells, persuading them to produce nerve toxins that disrupt my brain’s neurotransmitters.

In short, Lyme has made me tired, listless, depressed, and stupid.

This is a fascinating experience. Or, at least, it would be fascinating, if the symptoms themselves didn’t prevent me from caring. Other than short bouts while ill—when I’ve actually had the flu—I’ve never been listless or unmotivated. Quite the opposite, in fact. I’ve never been depressed. And I’ve never been stupid. I’m pretty much neurotypical.

So I want to explain, briefly, the bits about depression and stupidity, if only to capture the experience for Future Waldo.

Depression isn’t at all what I thought it would be like. I don’t feel depressed, by which I mean that I don’t feel sad or despairing or anything like that. I just feel less. Mostly, I feel like I couldn’t be bothered. Left to my own devices, I probably wouldn’t eat much, if anything. I doubt that I’d shower or shave. I’d mostly watch television and nap. My preferences are largely gone. (Should we have chicken or fish for dinner? I just don’t care.) I have a very short attention span; like a puppy chasing a butterfly, I’m happy to pursue whatever shiny thing presents itself, until a new shiny thing comes along. With substantial, headache-inducing effort, I can fake being normal-ish, but not for long. Depression, at least as I’m experiencing it, is the absence of emotion, rather than negative emotion. I don’t mind it, not yet, but maybe depression is what keeps me from minding depression.

Stupidity is also different than I’d thought. Part of my stupidity stems from the depression, I think. My curiosity is muted, my ambition to learn more or consider options more deeply has vanished. But part of it is just straight-up stupidity. I can’t really think about more than one thing at once. When a new thought enters my head, the old one simply vanishes. (Thanks to the depression, though, I don’t really mind.) I have no critical thinking skills, little ability to string together a cohesive argument, and a poor recall of long-held facts. I’ve been stuck at this point in this paragraph for at least ten minutes, unable to think of the other ways in which I’m stupid, or to spend more than five consecutive seconds trying to think of them. Yes, I’m too stupid to explain how I’m stupid. Do me a favor and pause to let the irony of that sink in, because if I pause, I’ll start forget what I’m supposed to be thinking about.

The good news is that this isn’t permanent. I started on an aggressive, two-week round of antibiotics yesterday, and I intend to find an infectious disease specialist with experience in Lyme to chart a more aggressive path to wellness. In theory, once the antibiotics start to kill off those spirochetes, I’ll stop feeling sick and stupid. (Unfortunately, the antibiotics will also kill off many of the bacteria that my body needs, so I’ll be having lots of homemade pickles and sauerkraut, coincidentally ready to eat this week, plus yogurt and kimchi, which will help to repopulate my gut’s microbiota.) I don’t know how long it’ll be until I start to feel better. Again, thanks to the depression, I don’t really care, although I know that I’m supposed to.

I hope that, in retrospect, this will have been a positive experience. It’s very difficult to understand how somebody else’s brain works. It’s hard to sympathize with those who did less well in mental aspects of that great genetic lottery, because usually one can’t really know what another person’s experience is like. Learning to understand depression and a different level of intelligence is a rare opportunity, and I’m optimistic that this is an chance to become a better person.