There’s a lot about an outsourced scrum team that is attractive to government agencies, but the most appealing bit is usually this: instead of writing a bunch of requirements up front, you can write them as you go, in the form of user stories. That’s an easy sell. A harder sell, though, is another change that must accompany this: you cannot pass your agency’s legal, regulatory, and policy requirements straight through to vendors and say “you execute these”—the agency needs to understand those needs and express them in the form of acceptance criteria. This shift is frightening to agencies, and can be a deal-breaker.
Under an outsourced Agile model, the vendor’s scrum team performs user research, which results in user stories. The product owner (a government employee) edits and prioritizes the user stories in the backlog, writing acceptance criteria for each user story, which gives the agency constant oversight and control over the exact work being done. Then each sprint starts with the vendor’s scrum team pulling user stories from the prioritized backlog.
We take this approach to have lowercase-A agility in our work. After all, if we put all of these tasks into the contract, then the vendor would have to complete them, regardless of whether they turn out to be valuable to users. An Agile model gives us the flexibility to adjust work as we go based on what we learn. In fact, we put as few requirements in the RFP and contract as possible, because if any work is specified in the contract, it has to happen. So the requirements that we include are generally non-functional requirements like documentation, durability, and quality, plus a few like the programming language to be used or the cloud hosting environment that the software will live in. That list of requirements should require just a few sentences.
But if the contract includes any specific tasks that the vendor has to do, now we’ve got a mess on our hands. That’s because now the vendor will have to, at times, ignore what the product owner says, ignore the backlog, ignore user stories, and just do some other stuff because that’s what’s in the contract. If the contract says “the vendor shall integrate the new system with the legacy system,” and the vendor isn’t confident that there will be user stories that direct them to do that, then the scrum team will, at some point, declare “sorry, we’re going to completely ignore all user stories for as long as it takes us to perform the integration required by the contract.” That may not come at a time that is convenient for government, and that work may not be done in the way that the agency would choose, but it doesn’t matter because the vendor must comply with the contract. When you put such requirements in the contract, you’re tying your hands, not your vendor’s.
My interest here is not these specific tasks, though—it’s instead in the practice of incorporating complex policy documents by reference and saying “oh, also do all this stuff.”
It’s normal for agencies’ RFPs and professional services contracts to require that the software comply with agency-specific, local, state, and/or federal policy documents about privacy, security, and technical practices. Often these are arcane, agency-specific policies that would be meaningless to anybody but the vendors already performing work for that agency.
More often these documents are so vague as to be useless. What does it mean for software to “comply with HIPAA”? What does it mean for software to be “in compliance with NIST 800-53”? Such requirements are meaningless nonsense, but the larger problem is that passing along these binder-sized sets of requirements necessitates that the scrum team become experts in those binder-sized sets of requirements, instead of actually doing the work that you hired them to do. Then, after they become experts in those, they will have to apply this knowledge by ignoring what the agency product owner tells them to do, and instead doing what they think the binders say to do.
The user research might make clear that the system’s users (say, unhoused people) cannot reliably have access to a consistent mobile phone, but if the agency’s policy documents say “the vendor shall use SMS-based two-factor authentication,” then that’s what the vendor has to do, because the contract says that they must follow the agency policy documents. Incorporating these policy documents into the contract means that whenever there’s a conflict between user needs and the policy documents, the vendor must resolve them in favor of the policy documents; the agency will have no say over this. “Vendor must comply with Agency’s Technical Standards document” requires that the vendor be an expert in your agency’s legal, regulatory, and policy requirements, which is a waste of the scrum team’s time, it’s expensive, and it’s profoundly anti-Agile.
This whole process is terrible. But what’s to be done?
The solution is straightforward: have the agency incorporate the relevant bits of the various policy documents in the acceptance criteria of user stories, not the contract. When the product owner encounters a user story that implicates medical privacy, she should pull in the agency’s HIPAA expert to figure out what acceptance criteria are needed to ensure that the user story is in compliance. When the product owner encounters a user story that implicates security, she should pull in a representative from OCIO who can figure out which NIST 800-53 controls are relevant, and turn those into acceptance criteria. And so on.
The bottom line: If you’re going to hire a vendor to bring an Agile development methodology to a project, you cannot expect to both have fine-grained control over the work they’re doing and also hand them a pile of policy that they’re obliged to untangle and implement. If you make your Agile vendor be an expert in how your agency does stuff, you will only get the exact same vendors that you’re getting now. And how’s that working out for you?