On 6 May 02013 with 2 comments
Since creating Richmond Sunlight and Virginia Decoded, I’ve been building up a public trove of datasets about Virginia government: legislative video, the court system’s definitions of legal terms, court rulings, all registered dangerous dogs, etc. But they’re all scattered about on different websites. A couple of years ago, I slapped together a quick site to list all of them, but I outgrew it pretty quickly.
So now I’m launching a new site: the Open Virginia data repository. It’s an implementation of the excellent CKAN data repository software (which will soon drive Data.gov). The idea is to provide a single, searchable, extensible website where every known state dataset can be listed, making them easy to find and interact with. It’s built on the industry’s best software, in part because I’m hopeful that, eventually, I can persuade Virginia to simply take the site from me, to establish a long-overdue data.virginia.gov.
There are a few new datasets that accompany this launch:
- The Dangerous Dog Registry as JSON, meaning that programmers can take these records and do something interesting with them. (Imagine an iPhone app that tells you when you’re close to a registered dangerous dog.) Previously I provided this only as HTML.
- VDOT 511 Geodata. This is the GeoJSON that powers Virginia 511, exposed here for the first time. Road work, traffic cameras, accidents—all kinds of great data, updated constantly, with each GeoJSON feed listed here.
- Public comments on proposed regulations. Over 28,000 comments have been posted by members of the public about regulations to the Virginia Regulatory Town Hall site over the past decade. Now they’re all available in a single file (formatted as JSON), for programmers to do interesting things with.
There’s so much more to come—good datasets already available, and datasets that need to be scraped from government sites and normalized—but this is a good start. I’m optimistic that providing an open, accessible home for this data will encourage others to join in and help create a comprehensive collection of data about the Virginia government and its services.
On 28 March 02013 with 10 comments
Our sweet old beagle passed away today.
It seems like it was just yesterday that we took her in. Back then—in late 2006—she was just another foster. My wife and I decided that, rather than fostering lively animals and finding homes for them, we’d pick out the most pathetic dog we could find at the SPCA, as our very own Pygmalion tale. This elderly, frightened old girl was the clear winner, so we took her with us, determined to find her a forever home. There didn’t turn out to be much of a market for elderly, gassy beagles, and come March, we made things official.
Her life before 2006 was obviously pretty rough, and I’m proud that we could provide her with a great retirement home and a proper family. I hope all of her memories of her prior life were gradually replaced with a fuzzy impression of joy, safety, and freedom from want. She certainly provided us with many happy memories of our own.
I am one dog old.
On 23 March 02013 with 2 comments
It took just 27 hours for the $500 speech transcription bounty to be claimed. Aaron Williamson produced youtube-transcription, a Python-based pair of scripts that upload video to YouTube and download the resulting machine-generated transcripts of speech. It took me longer to find the time to test it out than it did for Aaron to write it. But I finally did test it, and it works quite well.
There are lots of changes and features that I’d like to see, and the beauty of open source software is that those changes don’t need to be Aaron’s problem—I (and anybody else) can make whatever changes that I see fit.
This will be pressed into service on Richmond Sunlight ASAP. Thanks to Matt Cutts for the idea, and to the 95 people who backed this project on Kickstarter, since they’re the ones who funded this effort.
On 21 March 02013 with 11 comments
The world needs an API to automatically generate transcript captions for videos. I am offering a $500 bounty for a program that does this via YouTube’s built-in machine transcription functionality. It should work in approximately this manner:
- Accepts a manifest that lists one or more video URLs and other metadata fields. The manifest may be in any common, reasonable format (e.g., JSON, CSV, XML).
- Retrieves the video from the URL and stores it on the filesystem.
- Uploads the video to YouTube, appending the other metadata fields to the request.
- Deletes the video from the filesystem.
- Downloads the resulting caption file, storing it with a unique name that can be connected back to a unique field contained within the manifest (e.g., a unique ID metadata field).
Rules
- Must be written in a common, non-compiled language (e.g., Python, PHP, Perl, Ruby) that requires no special setup or server configuration that will run on any standard, out-of-the-box Linux distribution.
- Must run at the command line. (It’s fine to provide additional interfaces.)
- May have additional features and options.
- May use existing open source components (of course). This is not a clean-room implementation.
- May be divided into multiple programs (e.g., one to parse the manifest and retrieve the specified videos, one to submit the video to YouTube, and one to poll YouTube for the completed transcripts), or combined as one.
- Must be licensed under the GPL, MIT, or Apache licenses. Other licenses may be considered.
- If multiple parties develop the program collaboratively, it’s up to them to determine how to divide the bounty. If they cannot come to agreement within seven days, the bounty will be donated to the 501(c)3 of my choosing.
- The first person to provide functioning code that meets the specifications will receive the bounty.
- Anybody who delivers incomplete code, or who delivers complete code after somebody else has already done so, will receive a firm handshake and the thanks of a grateful nation.
- If nobody delivers a completed product within 30 days then I may, within my discretion, award some or all of the bounty to whomever has gotten closest to completion.
Participants are encouraged to develop in the open, on GitHub, and to comment here with a link to their repository, so that others may observe their work, and perhaps join in.
This bounty is funded entirely by the 95 folks who backed this Kickstarter project, though I suppose especially by those people who kept backing the project even after the goal was met. I deserve zero credit for it.
On 17 February 02013 with 3 comments
Several times recently I have squeezed a large number of oranges, enjoyed some of the delicious fresh-squeezed juice, and then been disappointed by the rest the next day. It tastes bitter, and becomes worse rapidly. This turns out to be the result of naturally occurring limonoate A-ring lactone (aka "LARL," a tasteless substance) breaking down into limonin, which is very bitter tasting. The amount of LARL varies between oranges and throughout the growing season. If there’s any way to arrest the conversion of LARL to limonin in the home-squeezing process, I don’t know about it. →
On 4 February 02013 with 5 comments
My friend Jonathan Stray put together an entirely fact-based FAQ on American gun violence for The Atlantic. Everybody can learn something from this. →
On 1 February 02013 with 6 comments
I’m more interested in orange juice than is probably healthy for somebody who doesn’t work in the industry and, as such, I’m excited to see Bloomberg Businessweek shining a spotlight on the horseshit that is "fresh squeezed," "not from concentrate," and "all-natural." These are all lies. It was squeezed months ago. It was concentrated to a point a hair’s breadth from the legal definition of "concentrated." It’s not natural, it’s created in a lab in a process more complicated than Coca-Cola. If you drank the stuff as its stored in giant vats, you’d spit it out—it’s flavorless at best, disgusting at worst. It’s only through adding a cocktail of lab-created flavorings that it takes like something that came out of an orange. Because those lab-created flavorings are based on molecules that are found somewhere—anywhere—in nature, they can be labelled "natural flavors," instead of "artificial flavors." →
On 1 February 02013 with Comments Off
I’m more interested in orange juice than is probably healthy for somebody who doesn’t work in the industry and, as such, I’m excited to see Bloomberg Businessweek shining a spotlight on the horseshit that is "fresh squeezed," "not from concentrate," and "all-natural." These are all lies. It was squeezed months ago. It was concentrated to a point a hair’s breadth from the legal definition of "concentrated." It’s not natural, it’s created in a lab in a process more complicated than Coca-Cola. If you drank the stuff as its stored in giant vats, you’d spit it out—it’s flavorless at best, disgusting at worst. It’s only through adding a cocktail of lab-created flavorings that it takes like something that came out of an orange. Because those lab-created flavorings are based on molecules that are found somewhere—anywhere—in nature, they can be labelled "natural flavors," instead of "artificial flavors." →
On 21 January 02013 with 35 comments
Today was a big day for Senator Henry Marsh. The legislator of twenty years took a rare day off during the Virginia Senate’s 46-day session, to attend President Barack Obama’s second-term inauguration in Washington D.C. For the 79-year-old black civil rights lawyer, attending a black president’s inauguration on Martin Luther King Jr’s birthday is perhaps the most auspicious of occasions. Certainly nobody would object to him missing just one day. Looking at today’s legislative calendar, he would have seen that his absence wouldn’t be problematic, with nothing contentious on the agenda. (With the Senate split 50/50 between Democrats and Republicans, and with a Republican lieutenant governor acting as tie-breaker, that’s no small point.)
Marsh grew up under Jim Crow. He had a ten-mile round-trip walk to his one-room schoolhouse—an awfully long trip for a seven-year-old—while white kids took a bus to a modern school. Marsh didn’t let racism hold him back. He didn’t just graduate from primary school, but went onto college. When he was a senior at Virginia Union University, the Byrd Machine was organizing “massive resistance”—shutting down public schools rather than comply with Brown v. Board of Education—and Marsh got involved, testifying against the policy before the General Assembly. In doing so, he met famed civil rights attorney Oliver Hill; at Hill’s encouragement, he got a degree in law from Howard University, and later went into private practice with Hill, focusing on civil rights law. Marsh and his practice were responsible for huge advances in civil rights over the decades, eliminating “separate but equal,” busing, and racial discrimination in hiring. Along the way he became the first black mayor of Richmond, and was elected to his Senate seat in 1991. Today he chairs the Martin Luther King Jr. Memorial Commission and created the Martin Luther King Jr. Living History and Public Policy Center.
So it bears repeating: today was a very big day for Henry Marsh. He must have taken a great deal of satisfaction in seeing his life’s work culminate in the first black president’s reelection, being sworn in on Martin Luther King Jr’s birthday. It was a very, very good reason to miss a day’s session.
Today was also a big day for Senate Republicans. They knew that Henry Marsh would be at the inauguration today, and that the 20–20 split in the Senate would become a 20–19 split while Marsh was 100 miles north, among the throngs on the National Mall. So today was the day that they decided—without hearings, advertisements, notifications, or warnings—to take a chunk out of Marsh’s district, along with a handful of others, to ghettoize black voters in a majority-minority district and put 45% of voting-age citizens into new districts.
I sat in the Senate gallery, along with no more than perhaps a half-dozen other people, slack-jawed with confusion (tweeting all the while) as Republican Sen. John Watkins filibustered through the allotted 15 minutes to discuss what was advertised as the third reading of a pretty boring bill, making technical adjustments to district boundaries. Unbeknownst to anybody but the 20 Senate Republicans, the bill had been replaced with a radical redistricting, combining two senators into a single district (eliminating the district of 2009 Democratic gubernatorial nominee Creigh Deeds), reshuffling district boundaries throughout the state to absorb those changes (to Republicans’ apparent favor in a half-dozen districts), and creating a “black district.”
Senate Democrats tried repeatedly to get a word in, but they were blocked procedurally. A series of votes were held (votes about voting, votes about reconsidering voting about voting, and so on), all failing 20–19, during which a few people got to make remarks. One Democratic senator moved to simply put the vote off until tomorrow, so that there’d be time to read this brand-new bill. That vote failed 20–19. Another Democratic senator pointed out that this was simply unconstitutional (“[t]he General Assembly shall reapportion the Commonwealth into electoral districts in accordance with this section in the year 2011 and every ten years thereafter”). One Republican senator insisted that this was simply a racially sensitive improvement, since it was establishing a majority-minority district. Another Republican said that there was no need to hold hearings on this new redistricting, because they held hearings a few years ago, last time they redistricted. Yet it remained unclear throughout what, exactly, this bill did, though Democrats were frantically trying to figure that out as they stalled with round after round of procedural vote, a peeved Lt. Gov. Bill Bolling presiding over the whole affair. Finally there was nothing else to be done—the vote was held, and the bill passed, 20–19.
Lt. Gov Bolling says he would have voted against the bill, if it had been a tie. Which is surely why the bill was introduced today.
Senate Republicans’ MLK Day gift to Senator Marsh and to Virginia is to use the re-inauguration of the United States’ first black president as cover to pass a bill that will make it harder for black candidates to get elected.
Now the bill goes to the House of Delegates, who will no doubt pass it, and then to Gov. Bob McDonnell, who said he was as surprised by this bill as everybody else. We’re about to learn if McDonnell has really become the centrist he’s presenting himself as, or if he’s the same old right-wing extremist. I fear we already know the answer.
On 14 January 02013 with 1 comment
Physicist Richard A. Muller was in the news last year after his Koch-funded study of global climate change concluded that it’s real, surely to the Koch brothers’ dismay. Now he’s penned an on-ed for the New York Times in which he says that his ongoing research has led him to the same conclusion as 99.9% of other experts in the field—that "essentially all of this increase results from the human emission of greenhouse gases." Muller’s research shows that the UN and the IPCC actually understate the problem. He researched the climate change causes claimed by non-scientist skeptics (urban heating biases, cherry-picking data, faking data, solar activity, and global population), and found that none of them explained climate change. What did explain it perfectly was atmospheric carbon dioxide. →
On 14 January 02013 with 6 comments
The NRA’s major constituency isn’t their individual members, but gun manufacturers. We are entering the era of 3-D printers (I have several friends who own them), and it’s entirely possible to print a gun. Including counterfeit guns. The stock, the barrel, the receiver—everything. Adam Penenberg explains how this is liable to to have the NRA begging congress for regulation of the indstury, rather the opposite of the present situation. →
On 27 December 02012 with 4 comments
The etymology of “the whole nine yards” is a total mystery. Anybody who tells you that they know its origin is either lying or unknowingly parroting an urban legend. The number of feet of fabric required to make a suit? Number of cubic yards of soil removed to dig a grave? Number of cubic yards of cement that fits in a mixer? The length of a WWII-era ammunition belt? Nope, none of those are it. The earliest known use of the phrase was in 1962, but now there’s been a trio of new discoveries from 1921 and 1912. Why weren’t they found before? Because the phrase was “the whole six yards.” The number was inflated over the years, much as “cloud seven” became “cloud eight” and is now “cloud nine.” The origin of the phrase is still unknown, but one potentially important clue is found in the pair of 1912 uses—both were in Kentucky. →
On 10 December 02012 with Comments Off
For those of you not quite clear on what it is that I do, you might be interested in an article that I wrote for the Cornell Law School’s Legal Information Institute, explaining The State Decoded and its application to laws and regulations. The project is more than halfway done, and I’m pleased with its progress. →
On 8 December 02012 with 12 comments
When a candidate is described as “divisive,” generally it’s intended to mean that while his own party loves him, the other party can’t stand him. In what’s shaping up to be a race between Terry McAuliffe and Ken Cuccinelli for the Virginia governorship, there are two wildly divisive candidates who are perhaps more divisive within their own parties than outside of it.
Four years ago, McAuliffe came in a distant second in a three-man race for the nomination for governor (despite raising $8M), won no geographic portion of Virginia, and endeared himself to nobody in the process. He’s never been elected to public office and has no constituency. The percentage of Democrats who would definitely not vote for him exceeds the percentage who would vote for him. That’s not in the primary—that’s in the general election. McAuliffe is a Clinton-era Democrat, the sort of old-school Democrat accustomed to winning elections by sucking up to power brokers, the sort who was purged from positions of power in the party round about 2005. It’s his turn to run for office, you see. He’s a glad-hander (it’s always “good to see you,” never “good to meet you”), always ready with the grip-and-grin. His performance at the 2009 Jefferson-Jackson Day Dinner really said it all.
And the Bill Clinton thing. Good Lord, the Bill Clinton thing. Guess who McAuliffe just got off the phone with? Guess who he just played golf with? You know who told him the funniest thing the other day? McAuliffe cannot stop mentioning Clinton because it’s all he’s got. Terry McAuliffe : Bill Clinton :: Marge Simpson : Chanel suit.
McAuliffe’s business bona fides aren’t much better. Global Crossing. (Need I say more?) His current business, Greentech Automotive, recently established an auto plant…in Mississippi. Despite that McAuliffe knew full well that he’d be running for governor of Virginia, touting his business experience on a platform of creating jobs. (“The main reason Terry is running for Governor is to make it easier for companies to create jobs right here in Virginia,” says his spokesman.) Why didn’t he build the plant in Virginia? Oh, it’s not his fault—it’s the fault of Virginia Economic Development Partnership! “It was their decision,” McAuliffe spinelessly informed The Note. How was the location of his factory a decision of tiny state agency? They wouldn’t pay him enough to locate his plant in Virginia. Yes, McAuliffe believes that states should bid for businesses (an economic loser just about every time), even his own business, not by creating environments conducive to running businesses and recruiting employees, but by just offering cash. This, of course, is why this company was located in Hong Kong when he bought it—in a globalized economy, a lowest-bidder approach will leave manufacturing out of the U.S. permanently. Mississippi, you see, is the U.S.’s version of a third-world country. Perhaps McAuliffe will be running on a platform of making Virginia more like Mississippi? In their defense, VEDP says that McAuliffe never even completed their application. McAuliffe shopped around for a state in which to open a factory the same way that he shopped around for a state in which to run for governor. Virginia’s apparently great for running for governor, but not so great for building “cars” that are legally identical to golf carts.
Not one Democrat in a hundred is excited about McAuliffe. Democrats are fired up about him the same way that Republicans were fired up about Mitt Romney. The base will fake it through November and, if he loses, they’ll all say how they never really liked him in the first place. If he wins, of course, they always believed in him!
Then there’s Ken Cuccinelli. Christ, what an asshole. “Extremist” has seldom been a more suitable word to describe a candidate. He supports a no-exceptions ban on abortion, opposes homosexuality (period), thinks Virginia needs Arizona-style anti-immigration laws, believes that global climate change is a conspiracy theory, and thinks that President Obama didn’t really win reelection last month. He has 95% of the traits that have been laying waste to the Republican Party in recent years, save one—he’s not dumb. In fact, he’s an intelligent guy, and a too-common mistake made by Democrats is to believe that just because he professes wildly retrograde, utterly contra-factual beliefs, that he must be a fool. He is not. (This is in sharp contrast with Sarah Palin, Rick Perry, Michele Bachmann, etc., who, even collectively, are dumber than a sack of hammers.) Unlike McAuliffe, he actually has a base, and he’s been elected to office repeatedly. He represented Fairfax in the General Assembly for two terms and, of course, successfully ran for attorney general. While McAuliffe is a generic sort of a centrist-ish Democrat who is hobbled by a terrible personality and the perception that he’s a carpetbagger, Cuccinelli is hobbled by holding views that are wildly out of step with Virginians, Americans, and the facts. I’d guess about 20% of the electorate probably adores him, but far more deplore him (or will, come next October).
Cuccinelli is the sort of social conservative that’s driving a wedge into the Republican Party. Regular ol’ fiscally conservative Republicans have tolerated allowing this type into the tent so long as they’ve furthered the same collective goals, but it’s started to get embarrassing (e.g., the Tea Party). Those regular Republicans were victorious in the nomination process for the presidency this year, allowing Romney to defeat a field that consisted largely of crazies, but after Romney’s loss, it’s not clear which side will be running the party soon. Dick Armey’s departure from FreedomWorks (one of the the Koch-funded companies that created and bankroll the Tea Party) is the latest evidence that the conservative power brokers have lost control over their own creations—the inmates are running the asylum. Cuccinelli is proudly on the inmates’ side of the fence, and unless he’s prepared to tamp down that image, he’s going to have a tough time getting support from the kind of Republicans who supported Mark Warner over Jim Gilmore. This, of course, is why Lt. Gov. Bill Bolling is flirting with running as an independent. His is the wing of the party that thinks that the grassroots need to be trimmed back (to abuse a metaphor), the wing that Cuccinelli is going to have a tough time wooing, and a tougher time still if Bolling gets into the race.
Power-brokers on both sides are pooh-poohing talk of primary challengers and third-party candidacies. When former congressman Tom Perriello demurred a few days ago, that was the prompt for Democrats to declare that it’s time to get behind McAuliffe as our candidate. This has not been greeted with enthusiasm.
All of this reminds me of the Republican presidential nomination process in 2008 and 2012. Reviewing every candidate, there was a clear and obvious argument to be made as to why they couldn’t possibly win the nomination. And yet somebody had to win and, indeed, somebody did. Neither Cuccinelli nor McAuliffe can possibly win a gubernatorial election. And yet—unless somebody else enters the race—one of them will.
This is the worst kind of election, the kind in which a supermajority of the voters in each party have to support not their preferred candidate, but the one whom they loathe the least. (To be fair, this is how some voters feel about every election.) That may be what makes it such an ideal race for a solid third-party candidate like Bolling to take a run at election. Russ Potts’ 2005 gubernatorial candidacy was a threat to Republican nominee Jerry Kilgore, but there was never any danger of him winning the election. Republicans were OK with Kilgore, and Democrats liked Tim Kaine. Things are different this time. If Bolling can trim his sails a bit (he is a conservative Republican after all), he can take votes from both candidates, money from both sides, and I think it’s entirely possible for him to win. At least, then, it’ll be possible for somebody to win.
On 4 December 02012 with Comments Off
In a long piece, the New York Times looks at Michigan’s efforts to persuade Hollywood to make films there through enormous subsidies. It didn’t fail—worse, it succeeded. The state lost money hand over fist in the process. Paying businesses to relocate to your state or city is a huge waste, 99% of the time. →
On 26 November 02012 with 6 comments
The prior chair of the Florida Republican Party (2006–2010) and former governor Charlie Crist (2007–2011) have both told the Palm Beach Post that they didn’t push voter ID laws and the like in order to reduce fraud (there is none)—they did it to suppress turnout. The more people voting, the better Democrats do. They invented the fraud concern as a “marketing ploy.” Both men are on the outs with Florida Republicans, and this probably isn’t helping them make up. →
On 17 November 02012 with 1 comment
We have gone 332 months in which every month has been above the historical average temperature. There hasn’t been a single colder-than-average month since 1985. →
On 14 November 02012 with 2 comments
So says Republican leaders in Georgia. The UN! Agenda 21! Rural broadband! RAND! Obama’s 2008 election yielded the Tea Party. His reelection may breed an even stupider brand of nutjobs. →
On 13 November 02012 with 3 comments
A press release from Gov. Bob McDonnell’s office one year ago contained this quote:
If we’re going to lead America’s economic recovery, we have to remember that small business, not big government, creates jobs.
That’s a familiar refrain, heard often during the recent election, most frequently uttered by people seeking a job leading that very government that cannot create jobs.
A press release from McDonnell’s office today:
“The opening of the 495 Express Lanes means opportunity for Virginia,” said McDonnell. “The project not only helped create jobs during construction, but will continue to make Northern Virginia a more attractive place to work and live.”
Last year, McDonnell told us that government couldn’t create jobs. But today, he tells us that, it could and it did. I wonder what happened in the intervening year?
On 12 November 02012 with 3 comments
Thank God. What an embarrassment it would have been for Virginia if this casually vicious racist had won. He fashioned himself in the image of the unreformed old southern racists of the 1960s, and never bothered to update his mindset, only how to fake it for brief stretches. In the meantime, Virginia—and the whole US—changed around him. Maybe he can spend even more time now fighting against anti-Semitism, since he told Wolf Blitzer in 2006 that he’d “use [his] time on Earth” to do just that. I wonder how that’s been going in the intervening six years? →
