Procurement smells.

Major government software procurements fail at a high rate. There are a lot of methods of reducing the odds of failure, but how do you know if that’s necessary?

Developers talk about “code smells”—small things that are off in ways that indicate that there may be larger problems. So, too, are there procurement smells—the little tells in requests for proposals and associated materials that vendors use to know if this is a project they want to have anything to do with. We can use similar procurement smells to determine if a project has any real chance of success, or if it will join the ranks of the supermajority of major procurement that don’t pan out.

It works like this: Review the list of statements below, and tally one point for each statement that describes the project in question. Cumulatively tally similar statements, so that e.g. a 1,000 page RFP would be awarded a point for being 100 pages, a point for being 500 pages, and a point for being 1,000 pages.

  • The RFP is longer than 50 pages
  • The RFP is longer than 100 pages
  • The RFP is longer than 500 pages
  • The RFP includes 10 requirements
  • The RFP includes 50 requirements
  • The RFP includes 100 requirements
  • The contract is for both technical services and non-technical services (e.g., both building software and running a help desk)
  • The contract is for customized COTS
  • The contract is for more than $10 million
  • The contract is for more than $50 million
  • The contract is for more than $100 million
  • The contracting officer insists on “one throat to choke”
  • The agency has no intent or capacity to inspect the vendor’s code
  • The RFP says nothing about user research
  • The contract is fixed price
  • Work is to be delivered to government only at the conclusion of the performance period
  • The period of performance exceeds three years
  • The vendor will own the resulting software

OK, now score your project!

0 points: This could work
1 point: This is not impossible
2 points: You’re going to have a bad time
3 points: Get out while the gettin’s good
4+ points: Abandon hope, all ye who enter here

(To learn how to avoid these problems from happening in the first place, see 18F’s De-Risking Guide, particularly “Budgeting and overseeing tech projects.”)

Have any additions to propose? Comment here, email me, or reach me on Twitter

The disconnect between software development and government contracting.

There’s a big disconnect between modern software development practices and government contracting. It can seem intractable, but there is a solution.

It’s the job of contracting officers to get government the best value for their money. That means being sure that they’ll get precisely what they need, within budget and on time. Normally, the best way to do this for a “firm, fixed price”: buying X gizmos that meet Y spec for $Z.

Firm fixed price (FFP) is by far the preferred method of procuring stuff. And for good cause! Contracting officers are taught that nearly any other contract type (i.e., labor-hour, cost reimbursement, or time and materials) is more risky. And that’s generally true.

That’s what has led us, for decades, down this path of wildly over-specified software RFPs. FFP for complex software means you need to have hundreds of pages of specs, because you’ve got to specify exactly what you’re getting for your money. That’s seen as safe.

In the meantime, software developers gave up that kind of software development and moved to Agile. We know that if you define all that stuff up front, you’re sunk. You’ll build something bad that the users hate, but that sure does meet the spec.

This has left a fundamental disconnect between contracting practices (specifying everything is safest) and software development practices (specifying everything is dangerous).

The solution to this is the obvious one: contracting officers need to stop using FFP for software development. There is literally nothing you can FFP for Agile software development. (See my article about doomed efforts to FFP story points.)

The way I’ve sold this to contracting officers is with this point: When buying custom software, you’re not buying a product. You’re buying a service, the service of developers building software, with features as prioritized by a government product owner.

To many contracting officers, this is nonsense. But to the right contracting officers, they get it. They see how Agile + a time & materials contract allows for extremely close oversight, tight control, and severability for non-performance. It’s safer than FFP.

There’s no way around the disconnect between contracting officers wanting FFP contracts and how software is actually created in the 21st century. The solution is to tackle the problem head on, to educate contracting officers, and let the solicitation & contract reflect the work.

To learn more about this, I recommend “Use time and material contract types for custom Agile software development services” in 18F’s “Federal Field Guide.”

Never contract for story points.

When contracting for Agile software development services, sometimes contracting officers make “story points” the thing that they’re buying. This is an enormous mistake, on a couple of levels, and nobody should ever do it.

Let’s talk about why.

First, let’s define “story points.” Agile development teams need to figure out what they’re going to work on at the start of every ~two-week sprint. Working with the product owner, they need to identify a bunch of user stories that will take as close as possible to two weeks.

One way to figure this out is for the team to figure out how many “story points” to assign to each story — basically, how much work it will be to get it done. That number might be labor hours, or it might represent how complex or uncertain the work is. There are all kinds of ways to come up with story points. Story points are a rough number, sometimes picking from a number in the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21…) to make that clear.

The team might have learned that they have the capacity for e.g. 300 story points in a sprint, so they’ll pull in a total of about 300 story points of work. The team decides these numbers. They are only for the purpose of estimating how much they can do in a single sprint.

Contracting officers hate time and materials contracts. They want to contract for a firm, fixed price (FFP). When procuring Agile development services, they desperately look for something, anything, to FFP. (Hint: There is nothing.) So they sometimes settle on story points as a thing that they’ll buy for a fixed price per point.

The first problem with contracting for story points is that story points are imaginary. They are a tool used by the development team for the development team. They are sometimes externally indefensible, which is fine, as long as they work for the team!

The second problem with contracting for story points is that they are, basically, a fictional currency with a value invented by the vendor. You might as well be buying 1,200 flibberdigibbets, or 1.6 million dinglehoppers. The value means nothing, can change at any time, and cannot possibly be fixed within a contract. (If you’re thinking “you could fix them at one labor hour per story point,” then, congratulations, you have invented labor hour contracts, which is precisely what contracting officers are trying to avoid by using FFP contracts.)

In short, if a contracting officer knows anything about about a vendor’s story points, something has gone terrifically wrong.

For more about the proper methods of procuring and overseeing Agile development services, see “Oversee Agile projects by measuring end user outcomes instead of requiring project teams to perform tasks by specific dates,” in 18F’s Federal Field Guide.

Reduce bids by reducing uncertainty.

When government agencies procure custom software, the price tag is often driven up because the agencies are unwilling or unable to reduce the complexity prior to beginning the acquisition process. The complexity and associated uncertainty is obvious to vendors, so when asked to provide a firm fixed price bid, they’re going to price it for the worst-case scenario, to avoid losing their shirt.

If agencies would allocate the resources to simplify and de-risk the work prior to publishing a solicitation, they’d receive substantially lower quotes from vendors, who would no longer be required to price in uncertainty.

Let’s use a metaphor here.

Imagine that I find that my finished basement has a wall that is wet and has a black substance growing on it. So I ask a half-dozen contractors for fixed-price bids to fix the problem. Those contractors have to assume a worst-case scenario: the basement wall is collapsing, and has cracked enough to admit water, which has resulted in black mold (Stachybotrys chartarum). That’s going to require demolishing the interior wall (subcontracting for mold remediation), digging out many cubic meters of earth, jacking up the house, tearing down the old wall, pouring a new wall, re-framing the interior wall, putting on new drywall, and painting. That’s going to cost tens of thousands of dollars.

But imagine that, instead, I do a little homework first. I take a sample of the black substance and pay a lab $50 to test it, and they tell me it’s just mildew. Then I tear off the wet drywall and find there’s a white PVC pipe that has a slow drip coming from a connector. Now I know that I really need two things: a plumber to cut out the connector and put in a new one (that’s an hour’s work), and a contractor to replace the ruined sheet of drywall and repaint (that’s maybe two hours of work). Now I can ask a plumber and a contractor for bids, and those bids are going to be way cheaper, since I’ve eliminated nearly all of the uncertainty associated with the work. And I know enough about the problem to have an intelligent discussion with a contractor about their proposal. Instead of $30,000, the bids will total maybe $400.

Agencies generally lack the knowledge about how to do that homework. They don’t employ people who understand software development; or, if they do, those people are not consulted prior to starting a procurement. The contracting officers don’t know about software development, and the software developers don’t know about contracting. Those two experts sure wouldn’t be expected to work together, or even know each other. So agencies pay wall-rebuilding rates for pipe-repair work.

Government agencies at all levels could better control spending, improve their budgeting processes, and take charge of their technology-intermediated service delivery, if they would simply allocate the resources to understand what they need to procure.

Government should procure custom software as open source.

Government software becomes vastly better when it’s procured as open source.

Normally, government buys closed-source custom software. Government never looks at the source code. The public can’t inspect it. Is it any good? No, it is not. There is no incentive to make it good. In fact, there’s a perverse incentive: hard to maintain means they keep the contract.

It’s not literally impossible to have bad code and good software, but let us agree that these things generally do not go hand in hand. Everything works a lot better if the vendors are producing great code.

Declaring right up in the RFP that all work will be public domain, developed in an open code repository, has a lot of great effects. The first is that vendors that write garbage code will self-select out of the running. They just won’t bid. You’re left with competent teams.

The second benefit is that those vendors that do bid will put their best people on that work. The vendors will want their work to look good. Anybody can look at their output, now and forever, including potential clients and potential hires. That’s unusual!

The third benefit is that the best developers will want to work on these projects at these vendors. A rare opportunity for their work to be seen by the public! By future employers! By their peers! Normally they toil in obscurity, but not on an open source project.

The fourth benefit is that it will be vastly easier to switch vendors (if need be), or hire new vendors to build onto this code base. Because it’s all public! They’ll know exactly what they’re getting into. That’s important because uncertainty is reflected in higher bids.

When everybody is working in the open, everybody’s interests are aligned: government, vendors, the vendor teams, and the public. When procuring closed source software, nobody’s interests are aligned. Government needs to stop doing that.

Vote ”yes“ on VA’s redistricting constitutional amendment.

Gerrymandering persists because it’s the rational choice for elected officials.

The way electoral districts are drawn in most of the U.S. is that state legislators decide what they want their districts to look like. The majority party that controls the legislature draws their own districts to allow them to cruise to reelection, using fancy redistricting software with household-level party membership data. And then they draw the minority party’s districts to make their lives difficult, such as by pitting incumbents against each other, lumping together unrelated communities, and making sprawling districts that are difficult to travel. This might sound complex, but the redistricting software makes it easy.

In short, legislators draw their own districts, ensuring their reelection. And why wouldn’t they, given the choice? It’d be completely irrational to do otherwise.

An illustration of how district lines can be drawn to allow a 60% "blue" / 40% "red" area to be divided into five districts allowing total control by either "red" or "blue."

Legislators generally get to do this once each decade, right after the decennial census determines how many people live where. The census is in 2000, 2010, 2020, etc., so redistricting is in 2001, 2011, 2021, etc.

Democrats and Republicans both gerrymander. It’s not an affliction of partisanship, it’s an affliction of power. Every time, the majority party defends their gerrymandering as mere redistricting, and the minority party decries the evils of gerrymandering and promises they’ll get rid of it when they’re in charge. This has gone back and forth for many decades.

Gerrymandering has lots of terrible effects. Especially the very premise: that legislators choose their constituents, instead of vice versa. There’s the effect on voters who are of the opposite party as their representatives, knowing that their votes will have no effect. Perhaps most important, there’s the resulting extremism. When the general election isn’t competitive, then the competition happens in the primaries. This results in the nomination of candidates who have no incentive to appeal to the opposing party, who get the nomination on the basis of promising to give no quarter to their opponents. This creates a spiral of extremism, and eliminates the possibility of bipartisan cooperation within the legislature.

In much of the U.S., there’s nothing that you and I can do about gerrymandering, because that requires a constitutional amendment…which legislatures must approve of. (Except in Delaware — they don’t bother with voter approval.) The majority party is quite happy with how redistricting is working for them, so they don’t pass those constitutional amendments. There’s nothing the public can do about it. (You might be thinking “well, people could vote them out of office.” No, they couldn’t. That’s the underlying problem here.)

So we’re stuck with gerrymandered districts.

* * *

In Virginia, right now, we have a chance to fix this, thanks to an extraordinary coincidence of timing.

Advocates for redistricting reform got to work a decade ago to get a constitutional amendment in place by 2021. With Republicans firmly in control of the General Assembly, it was easy to persuade Democrats in the legislature to get behind the cause. In fact, Democrats were so eager to support it that there was real danger that the movement would be perceived as a partisan one, so organizers had to go to considerable pains to avoid that.

In the 2019 session, the Virginia Senate was on the knife’s edge of control between Democrats and Republicans. Demographic changes in Virginia over the past decade had reached the point where Democrats had made big gains a few months prior, and Republicans were deeply concerned that they could find themselves a minority party by 2021. So the General Assembly was able to pass the legislation to amend the constitution to reform the redistricting process, by an 85-13 margin. Democrats supported it overwhelmingly. Sure, they watered it down, moving from nonpartisan redistricting to bipartisan redistricting, but it passed. But passing it once wasn’t enough — by law, they needed to pass the same amendment again the next year, after an intervening General Assembly election.

Before that could happen, though, something remarkable happened. Last November, Democrats flipped the House and the Senate, taking control of both chambers. Those new legislators were seated this year. And then the constitutional amendment came up for its second vote. By something had happened in the interim: Democrats were no longer so enthusiastic.

The same Democrats who cheerfully voted for the bill a year prior now had “concerns.” They wanted to “tweak” the legislation and perhaps delay when it would take effect, proposing substitute legislation.

But here’s the crucial fact about that substitute legislation: those changes would have reset the clock. That is, it wouldn’t have been the second time that the legislature had voted on this amendment, but the first time they had voted on a new amendment, requiring a pause of two years (so there could be an intervening legislative election) before the legislature could have the second vote. If it passed in 2023, and then voters backed it that November, that amendment would happen years too late for the 2021 redistricting, and have no effect at all until 2031.

Whatever the legitimacy of these newfound concerns, these Democratic legislators well knew that there were only two options: pass the amendment as it was, or vote against the amendment so they could gerrymander next year. There was no third option.

When forced to vote on the very bill they’d passed the prior year, what was the outcome? Well, every House Republican voted for the amendment, and nearly every House Democrat voted against it. Just nine House Democrats joined with Republicans in supporting it. (The Senate, on other hand, passed it 38-2, with two Democrats dissenting.)

Here are the 23 legislators who voted for the bill in 2019, and then voted against it this year:

  • Hala Ayala (D-51)
  • Betsy Carr (D-69)
  • Jennifer Carroll Foy (D-2)
  • Lee J. Carter (D-50)
  • Karrie Delaney (D-67)
  • Eileen Filler-Corn (D-41)
  • Wendy W. Gooditis (D-10)
  • Elizabeth Guzman (D-31)
  • Charniele Herring (D-46)
  • Patrick Hope (D-47)
  • Chris Hurst (D-12)
  • Mark Keam (D-35)
  • Kaye Kory (D-38)
  • Paul Krizek (D-44)
  • Mark Levine (D-45)
  • Kathleen Murphy (D-34)
  • David Reid (D-32)
  • Danica A. Roem (D-13)
  • Mark Sickles (D-43)
  • Marcus Simon (D-53)
  • Rip Sullivan (D-48)
  • Kathy Tran (D-42)
  • Vivian Watts (D-39)

(Of course, thanks to the intervening election, some legislators who voted for it in 2019 were gone by 2020, and some legislators were new in 2020. That’s the purpose of requiring an election between the first and second votes.)

When these Democrats were in the minority, they were all for redistricting reform. But now that they’re in the majority, one year later, now they’re against it.

As St. Augustine prayed, “Grant me chastity and self-control…but not yet.”

* * *

Gerrymandering persists because it’s the rational choice for elected officials.

Are legislators being hypocritical in voting for a bill and then voting against it? Yes. Are they behaving rationally? Absolutely. Left to their own devices, legislators will never vote to restrict their own power, only others’ power.

In 2007, then-House Minority Leader Ward Armstrong came to Charlottesville to speak at a public event that I held on redistricting. I asked him, before an audience of fifty or so Democrats, whether he supported redistricting reform. He delivered some relatively impassioned remarks about the importance of redistricting reform, about how we’ve got to get rid of gerrymandering. Then I asked him whether he’d still support that if Democrats controlled the legislature. He didn’t even pause before saying that, no, then he’d be against any kind of reform.

Extraordinarily, Virginia managed to get redistricting reform on the ballot, through an amazing coincidence of timing and, again, hard work by a small, dedicated group who pushed this for years.

Could we do better than bipartisan redistricting? Absolutely. Is bipartisan redistricting better than partisan redistricting? Absolutely. Should we let the perfect be the enemy of the good? Absolutely not.

Let’s make bipartisan redistricting a stop on the way to nonpartisan redistricting. Let’s move past the harms of gerrymandering, the extremism that comes of packed districts, the constant back and forth of each party punishing the other after they claw their way back from redistricting oblivion.

Let’s vote to pass amendment #1, making bipartisan redistricting the law of the land.

Make sure your UI modernization plan includes an open source clause.

Tens of millions of Americans have lost their jobs this year. Overall, over 35 million people have made unemployment benefit claims since the crisis began. State unemployment systems are crumbling under the load, and they’re desperate to modernize.

The CARES Act, passed March 27th, expands benefits for those who lost work due to the pandemic, notably $600 per week atop state-provided unemployment insurance (UI). It has an expanded scope, too — even people who were self-employed or worked in the gig economy can apply for Pandemic Unemployment Assistance (PUA) to receive that $600 per week. (At this writing, that funding expired a week ago, and Congress is wrangling over what to do.)

States’ legacy UI systems have struggled to keep up with the volume of applications, and are too inflexible to accommodate the changes in criteria.

At U.S. Digital Response, we spent April, May, June, and July volunteering at several state labor agencies, helping to patch struggling unemployment insurance systems. We’ve helped scale websites, troubleshoot call center volume spikes, and implement new web forms for PUA applications. As state labor agencies implement programs, they’ve also started revisiting their modernization plan, and in some cases, have asked us for advice and recommendations.

We’ve seen systems from all of the vendors in this space, from big firms like IBM, Deloitte, and Tata, to smaller firms like Fast Enterprises and Geographic Solutions. No matter what vendor states work with, we advise states to make sure they include an open source clause in their software development contracts. Here’s why.

* * *

State government requires custom software to operate. Unemployment insurance, Medicaid, DMVs, child welfare, SNAP, and all core government infrastructure depends on custom software bought by states at a large expense. Most of the software is overpriced and bad, but there is an easy way to improve this situation: open source software.

When it comes to important state infrastructure, top government software vendors use a double-dip business model. First, they charge us to build the software, then they charge us to use it. Nobody would pay to build a house and then pay to rent that house, so we shouldn’t do this with important state infrastructure. Vendors often claim that states are merely licensing software they’ve already built (“Commercial Off-The-Shelf,” aka “COTS”). This is often untrue. Instead, the software they’ve already built is custom for another state and will require large changes to work in a new state. These changes will be done at our collective expense, but with the resulting software owned by the vendor. In this scenario, we’re paying for a house to be rehabbed before we start to pay rent on it. This is absurd.

When paying a vendor to build custom software, vendors should be paid for their time, not for their time and for a license for the resulting software. The contract should reflect that, giving the agency ownership over the software. But it’s best to go a step further than that and place the software in the public domain, making it open source. Publishing government software as open source has many benefits with no drawbacks.

First and foremost, open source helps to prevent “lock-in,” in which a vendor tries to make it impractical or impossible for a client to ever switch to a competitor. When the source code is available for open replication, the vendor can’t use it as a weapon of monopoly. For future projects, vendors can inspect the existing source code prior to bidding, reducing uncertainty, and lowering the cost of the bids. This also reduces the switching costs associated with a system, which is important for government to be able to participate in a competitive marketplace.

Open source is also a crucial prerequisite for building secure software. Source code should never contain any secrets, like passwords, but it’s common for developers to have a casual attitude about this when the source code is closed. This is a mistake. Software can be decompiled — those secrets can be extracted. Source code can be leaked, as Edward Snowden demonstrated. And in many states, works of government are inherently in the public domain, which may include the source code that was thought to be secret. The Department of Defense maintains the excellent Open Source Software FAQ, and their answer to “Doesn’t hiding source code automatically make software more secure?” is summed up with the first sentence: “No.” Software projects should be open source from day one to avoid these problems.

Finally, the work done by developers will be enormously improved if it’s open source. Requiring all development to be performed on a social coding website like GitHub (think Facebook but for software developers) completely changes the incentive structure. Developers who work on closed source software generally work away by themselves — other than their immediate coworkers, nobody is ever likely to see their work. Government agencies rarely have anybody on staff who is capable of reviewing vendors’ source code to know if the work is any good, so they’re not inspecting their purchases. These are circumstances that provide developers with little reason to perform their best work. But if the RFP declares that the work will be open source, all of this changes. Vendors won’t want to respond if they know that their work is low-quality. Those who do respond will want to put their best team on the project. Those team members know that their work is being watched by friends and colleagues—and will be available for inspection by future employers forever—so their work is correspondingly high-quality. Open source gets the best work out of the best team at the best vendor.

How does an agency publish their procured software as open source? Simply, with a data rights clause:

Data Rights and Ownership of Deliverables — The Agency intends that all software and documentation delivered by the Contractor will be owned by the Agency and committed to the public domain. This software and documentation includes, but is not limited to, data, documents, graphics, code, plans, reports, schedules, schemas, metadata, architecture designs, and the like; all new open source software created by the Contractor and forks or branches of current open source software where the Contractor has made a modification; and all new tooling, scripting configuration management, infrastructure as code, or any other final changes or edits to successfully deploy or operate the software.

To the extent that the Contractor seeks to incorporate in the software delivered under this task order any software that was not first produced in the performance of this task order, the Agency encourages the Contractor to incorporate either software that is in the public domain or free and open source software that qualifies under the Open Source Definition promulgated by the Open Source Initiative. In any event, the Contractor must promptly disclose to the Agency in writing and list in the documentation, any software incorporated in the delivered software that is subject to a license.

If software delivered by the Contractor incorporates software that is subject to an open source license that provides implementation guidance, then the Contractor must ensure compliance with that guidance. If software delivered by the Contractor incorporates software that is subject to an open source license that does not provide implementation guidance, then the Contractor must attach or include the terms of the license within the work itself, such as in code comments at the beginning of a file, or in a license file within a software repository.

In addition, the Contractor must obtain written permission from the Agency before incorporating into the delivered software any software that is subject to a license that does not qualify under the Open Source Definition promulgated by the Open Source Initiative. If the Agency grants such written permission, then the Contractor’s rights to use that software must be promptly assigned to the Agency.

This can be dropped into an RFP or a contract as-is.

* * *

It’s trivial to commit to open source within an RFP, and the benefits are enormous. It reduces switching costs, improves security, and provides vendors with an incentive to do their best work. When states are procuring custom software, they should default to open source.

If you have questions about your unemployment insurance modernization process or need another set of eyes on your RFP, U.S. Digital Response is here to help.

How to stop failures of major custom software procurements.

When government pays companies to build big custom software programs for them, they succeed just 13% of the time. Here is why failure is so common, and about the simple change that turns those outcomes on their head.

Major government software projects fail because government has learned, over many years, to do exactly the wrong thing: specify down to the last pixel exactly how the software should look and function. All of those specs go in the request for proposals (or “RFP,” also known as the “solicitation” or the “tender”), telling the vendor exactly what to do.

(If you’re thinking “how is agile software development possible if you specify everything up front” then congratulations you already know the punchline.)

All those specs are created with no user research, with no regard for user needs.

The vendor who gets the contract is then required to make precisely the software outlined in that 800-page RFP, even if their own user research shows that it’s wrong.

This is, of course, madness.

Half of the major government software projects that “succeed” succeed only in the sense that they do what the RFP laid out. Was that what the users needed? Probably not!

The vendors who bid on this work know full well that it ain’t gonna work out. The staff assigned to these projects are staff who are OK with doing the wrong work that will probably never be deployed because what’s the point. It’s an industry built on an expectation of failure.

How do you stop these failures from happening? Pretty easily! Stop prescribing exactly what contractors need to do. Instead, state the outcomes that are desired and leave it at that. The 800-page RFP is now 20 pages.

Of course, this requires a government product owner overseeing work every day, this requires competent scrum teams doing work as prioritized by the product owner, and it requires a time & materials contract that pays vendors for their labor, not their software.

Basically, stop thinking of these RFPs as procuring software. You’re not buying a thing! You’re buying developers’ time, time in which they do work as prioritized by the government product owner.

That’s it! The contract is dead simple.

Is the vendor’s work not good enough? No problem — stop assigning them work and they’ll go away. You don’t even need to terminate the contract. Because they were using agile, you can hire a new vendor and them pick up where the old one left off. Done.

And that is the story of why major government software projects fail, and the simple change that stops that from happening.

For more, see the handbook that Randy Hart, Robin Carnahan, and I wrote about budgeting for agile software procurements.

My videoconferencing setup.

My job on a distributed team necessitates that I spend 1–6 hours meeting with folks via videoconference. Google Hangouts, Zoom, Appear.in, and GoToMeeting intermediate my professional interactions with co-workers, vendors, and clients. Initially, I joined these using my Macbook’s standard camera and iPhone earbuds, but spending so much time on the phone, I needed to make some changes. In the intervening six months, I’ve created a setup in my home office that I’m really happy with, at a total cost of about five hundred bucks.

Having a comfortable, flexible videoconferencing setup makes it easier to spend hours in meetings, but it also has improved substantially how I look and sound (and, presumably, how I’m perceived by others). These are the upgrades that I made.

 * * *

Camera

I am 6’3″ tall, meaning that I loom over my  Macbook camera threateningly. I also use a 23″ display with my Macbook, but displaying video on that screen means that I appear to be gazing over the heads of the other folks on the call.

The solution is an external, USB camera. Initially, I used a $30 Logitech standard definition camera (I can’t even find a model number on it). That addressed the matters of comfort and looking folks in the eye, but the video just didn’t look great.

I recently upgraded to a Wirecutter-recommended  Logitech HD Pro C920 ($62). The 1080p video is a huge improvement, even at the downscaled quality supported by most videoconferencing clients right now. The images are much sharper, clearly better than the Macbook’s internal camera.

Headphones

I don’t find EarPods comfortable for hours at a time. I also spend a non-trivial amount of time on voice-only calls. I was not happy with the Bluetooth earbud options out there, and just before I pulled the trigger on a pair of Plantronics, Apple started shipping AirPods ($160). I took a gamble on them, and I’m awfully glad that I did. They move seamlessly between my laptop and phone, they’re so unobtrusive that I often appear to be wearing no headphones at all, and the sound quality is fine for voice. They also have a built-in mic that’s OK, certainly better than that thing with EarPods where you spend half of the call holding the wire-attached mic up to your mouth.

I never realized how much time I spent subconsciously managing my EarPods cables until I switched to AirPods—my first time on a call with EarPods again seemed comparatively oppressive. The short length of EarPods’ cable kept me tethered to my display, keeping me from changing positions often. AirPods have provided freedom of movement while on calls.

Microphone

My webcam, my laptop, and my headphones all have built-in mics, but the audio quality is pretty low.

After some research, I went with that favorite of podcasters, the Audio-Technica ATR2100-USB ($62). I actually sprang for the the extra $17 to get the kit that comes with a boom arm stand and a pop filter. I have it mounted on the side of my desk, where I can keep it swung back toward the wall when I’m not using it, or pull it adjacent to my display when I’m on a call. I used to work in radio, so this setup is particularly comfortable for me.

This is perhaps the least-necessary upgrade for my videoconferencing setup, but I’m really pleased with the improved sound quality.

Lighting

Poor lighting is the Achilles heel of most people’s videoconferencing setups. That’s because what makes for good work lighting is really different than what makes for good lighting for video. In any call with more than a few people, at least one person looks like they’re recording a proof-of-life video. Window backlighting blows out the exposure, or single-source lighting casts their face in shadows. This is so important to us as a culture that it’s part of our language:  we describe unsavory people as “working in the shadows”; if you portray yourself badly, you “aren’t presenting yourself in the best light.”

I bought the Neewer 18″ LED Ring Light ($100), with a Fovitec StudioPRO 18″ Light Stand ($12). An 18″ ring light turns out to be enormous—perhaps a smaller one would do the trick. This has been transformative. It makes it look like I’m being interviewed on broadcast TV. The lighting is just excellent. This was the purchase that I was most dubious about, and the upgrade that I’m happiest with. I’m looking around for a swing arm that I could mount this on, though, so that I can get it off my desk and out of the way when I’m not using it.

Monitor Arm

This might not seem like it’s an important part of a videoconference setup, but it’s my experience that it makes a big difference. Having a screen-mounted camera means that being positioned properly within the frame requires that you stay in place relative to your monitor. This is uncomfortable and, ultimately, unhealthy.

Again, I went with the Wirecutter-recommended option: the AmazonBasics Display Mount Arm ($100). Throughout a long call, I can move my screen as I change positions at my desk, to ensure that I remain well framed in the video. This has been great.

 * * *

By way of comparison, here’s how I look with my old webcam:

Here’s how I look with my new webcam:

And here’s how I look with my new webcam with my ring light on:

The difference is striking.

Obviously, $500 is a lot of money for most people. I justify the expense as both recognizing that the reality of my future employment will involve lots of videoconferencing and that I’m saving a lot of money on commuting and my wardrobe.

If you’re going to acquire these one piece at a time, as I did, I recommend this order of priority: camera, light, monitor arm, microphone, headphones. This is also the order of impact, a good camera providing the most impact, the headphones providing the least.

Have alternate suggestions for equipment? Email me, tweet at me, or let me know in the comments.


June 2019 Update: I’ve been using Webcam Settings, the generically-named macOS program, and it’s become an indispensable part of my setup. Being able to adjust lighting settings in software is great, especially compensating for unwanted backlighting. Recommended.

Truth, earned credibility, and a publisher’s responsibility.

I spent much of the ’00s as a political blogger. I wrote here, mostly about state politics. When I decided to start writing about state politics, in 2003, I sought out other political blogs in Virginia. There weren’t many, maybe a half-dozen. I added them all to my blogroll, made a point of reading those sites and linking to them, and they did the same. Despite our often-oppositional political perspectives, our exchanges were friendly, informative, and fun. I’m still friends with those folks.

In the spring of 2006, I was casting around for how to elevate lesser-known Virginia political blogs, as it didn’t strike me as entirely fair that my site should get so much of the readership. So I set up a blog aggregator—a bit of software that would check each Virginia political blog’s RSS feed every half-hour or so, and syndicate all new blog entries on a central site, creatively named Virginia Political Blogs. It didn’t take long to set up, and having a central site to serve as a commons was immediately popular. Every blog entry was shown in the same context as every other, all in the same typeface, all on an equal footing, listed chronologically.

In the few months afterward, blogging exploded in popularity, in part because it became much easier to set up a blog. No longer was it necessary to install Movable Type, or the nascent WordPress—Blogger.com would host your site for free. For a while, in that lead-up to the 2006 midterm elections, there was a new Virginia political blog every week. In retrospect, this must have been the peak of the popularity of blogging, before the rise of Facebook and Twitter

Inevitably, the lowered technological bar meant that some less-knowledgable folks were able to participate in this commons. But that was OK, because this was a marketplace of ideas, and if people wanted to promote their foolish ideas, they could do that.

This went badly.

Any fool could start a website, get added to my aggregator, and immediately have an audience that numbered in the thousands. And fools did that, by the dozen. They didn’t have to earn an audience by writing things that other blogs would want to link to. They didn’t have to prove themselves in any way. When they wrote things that were completely wrong, offensive, or even dangerous, instead of going to an audience of a dozen people, it went to an audience that included not just every political reporter in Virginia and the DC metro area, but a great many crackpots and fringe-theorists as well.

By putting terrible ideas on even footing with great ones, and by replacing a free marketplace of ideas with a leveled one, I inadvertently created a promotional vehicle for the ignorant, the rage-filled, and the chronically dishonest. By presenting them all in the same, shared context, it gave them an aura of legitimacy. And by automatically reposting their occasional hate speech, violent imagery, and calls to violence, I was enabling—even endorsing—those things.

I knew what I had to do. In December, I pruned the worst actors from the list. They were enraged, and their rage was only magnified by their ignorance (“your taking my free expression that’s against the constitution!!!!”) and their partisanship (“classic libtard”). I’d created a commons, gave them access to it, and then took it away. They tried to create their own blog aggregator, as I recall, but it proved to be beyond their technological capabilities.

* * *
In 2016, my little mistake has been repeated as a huge, national mistake. We call it “Facebook.”

Any fool can create a Facebook account, get added to their friends’ feeds, and immediately have an audience in the hundreds, potentially in the millions. They don’t have to earn an audience with their writings, but instead rely on social obligations around relationships to be guaranteed a readership. Everybody’s Facebook posts are displayed in the same, shared context, with reshared, weaponized Russia propaganda adjacent to class reunion photos and New York Times articles.

Propaganda on Facebook has the aura of legitimacy. It’s been shared by a friend or family member, preying on our sense of trust. The name of the outlet that published the news is displayed in light gray at the bottom of the post, while the friend’s name and photo is displayed prominently at the top. News from eaglepatriot.co looks the same as any article from The Washington Post.

It only took me eight months to figure out that I’d inadvertently created a terrible system that was enabling dangerously stupid people. It’s been twelve years, and Facebook hasn’t learned that lesson yet.

I quite doubt that my poor editorial policy changed the outcome of any elections. Can Facebook say the same?