Digital Engineering/AI Product Assurance/AI Development Governance
Set the rules for how your team builds with AI
Every other assessment in this service line deals with software that exists. This one deals with what your team does next: the rules, guardrails and gates deciding how AI tooling is used in your delivery. We write them with you, pilot them and leave the artefacts.
The same fixes keep being needed because nothing changed upstream
A cleanup engagement ends well. Findings closed, tests added, the authorisation model made consistent, the dependency inventory written down at last. Six months later a second assessment finds the same classes of problem in code written since, not the same lines but the same classes.
Remediation is downstream. It repairs output and changes nothing about how output is produced, while the line runs faster.
Generation volume outgrew the review process.
A team that opened thirty pull requests a week now opens ninety, and the largest is 1,200 lines of plausible code no reviewer can hold in their head. Approval rates rise, attention per line falls, nobody decided it.
Every engineer has a private method.
One keeps a maintained context file. One pastes the failing test and asks for a fix. One uses an editor nobody else has. Three habits, three architectures, one codebase.
Agents make decisions nobody was asked about.
A migration is written and applied against a shared database. A package added to fix a formatting irritation brings eleven transitive dependencies. A failing test is made to pass by weakening the assertion. Each is defensible alone; none was a decision anyone made.
The record has gone.
Ask which of last quarter’s changes were substantially generated and nobody knows. That question now turns up in diligence and in questionnaires.
Velocity went up and confidence went down.
This is the sentence engineering leaders say to us. The team ships more than ever and the person responsible has quietly stopped being able to vouch for it: a governance problem in a code-quality costume.
What AI development governance covers
Five control areas, each written as something a team executes, not a principle to agree with.
Tooling policy.
An approved list of models, editors and agent frameworks, with a named owner and a route for adding to it. An approved list beats a ban: a ban moves the tool off the company laptop, not out of the codebase, and you lose sight of it. Where a team already uses something unofficially, and there almost always is, the policy adopts it under conditions rather than confiscating it. Then the rule engineers need daily: what may be pasted into a model. Customer records, production credentials, private keys and anything under a customer contract sit on the wrong side of it. Self-hosted or vendor-run matters at that boundary and nowhere else: it decides where data goes and what may be retained, not whether the code is good.
Prompting and generation standards.
Security-explicit prompting, referenced to the OWASP Top 10 and MITRE CWE weakness lists, so the defect class you are avoiding is named in the request rather than hoped for in the output. House patterns for what every session touches, namely authentication, authorisation, input validation, error handling and logging, delivered as context and rules files loading automatically in your team’s tools. Undirected generation drifts because a model has no memory of your other sessions and no opinion about your conventions; a context file is that memory.
Review discipline.
A scanner finds the committed secret and the vulnerable package. It does not tell you that a generated function bypasses the permission check three neighbouring routes apply, or that the retry it added turns a non-idempotent call into two charges. That is what human review is for, and the first thing to break at five times the volume. The failure mode is fatigue, not laziness: a reviewer who has approved eleven plausible diffs today gives the twelfth less attention, and the twelfth is the one that mattered. So review gets sized to reality: a rubric saying what to look at, a ceiling on diff size, an author obligation to state what was generated and verified. Ceremonial review is worse than none, because it produces a signature.
Automated gates in CI.
Static analysis runs before a human sees the change, so review attention goes on judgement, not findings a machine could raise. Dependency and licence gates block a package carrying a known critical vulnerability or a licence you cannot absorb. Secrets detection runs on every push and across history. Coverage carries a floor against changed lines, not the whole repository, because a repository-wide figure is the easiest number in engineering to game.
Provenance and traceability.
A durable record of which changes were substantially AI-generated, captured in commit metadata or a pull request field rather than by watching engineers work. It pays for itself in diligence, which now asks directly; in incident review, where knowing how a line arrived shortens it; and in licensing, where the origin of code is the argument. It is a property of the change, not a measure of a person.
Who this is for
Engineering leaders whose adoption outran their process.
The tools arrived, output doubled, and the release practice is the one written for a team producing half as much. You want the speed, and to sign your name under it again.
CTOs who have just been asked the question.
A customer, investor or regulator wants to know how AI is used in your development lifecycle and how its output is controlled. You would rather build the practice than write a paragraph you cannot defend.
Teams coming out of a cleanup.
You have paid once to fix generated software, through an engagement like a Vibe Code Cleanup, and would rather not pay again in nine months.
Companies scaling the team.
Habits that worked when three people knew each other’s code do not survive fifteen people generating.
When teams call us
A questionnaire arrived with an AI clause.
Not a security question answerable from the codebase, but a process question about how your team works, with a deal waiting on it.
An agent did something expensive.
It applied a migration, deleted a branch, or opened a pull request touching forty files when the task named two. Nothing catastrophic, and everyone knew it could have been.
A new engineer asked what the rules were.
Nobody had an answer beyond “use your judgement”, which is not a rule and cannot be taught.
An incident traced to a generated change nobody remembers reviewing.
The post-mortem finds the change but not the decision, because there was no decision.
The team is about to double.
Whatever practice exists is held together by four people talking to each other, and it fails at about the eighth hire.
Agent stop conditions
If you take one control from this page, take this one. An agent meeting an ambiguity resolves it, because resolving things is what it is for. Stop conditions decide in advance which ambiguities it may not resolve alone.
Scope boundaries.
The agent works inside a named set of paths and modules for a named task. Wandering into authentication, billing, migrations or the payment integration when the task was a form validation fix is a stop, not a bonus. Scope is stated positively, in terms of where it may work, because a list of forbidden places is never complete.
Irreversible actions.
Anything that cannot be undone by reverting a commit needs a human hand on it: deleting data, force-pushing, rotating a key, mailing real users, calling a third-party API that charges money or writes to another system. The test is not how dangerous it feels. It is whether git revert undoes it.
Credentials and infrastructure.
No agent reads production secrets, and none applies infrastructure changes to a live environment. It may generate a Terraform diff and explain it. A person applies it, and can say what it does.
Dependency addition.
Adding a package is a supply-chain and licence decision dressed as a one-line import. New dependencies stop for a human, and the reason for choosing it over something already present goes in the pull request.
Schema change.
Any migration is proposed by the agent, reviewed and applied by a person, with the rollback written before the forward path runs. In agent-heavy teams this is the most common source of expensive incidents we are called about, and the cheapest to prevent.
What escalation looks like.
The point is not that the agent stops working: a rule that only blocks people is deleted within a fortnight. It stops, writes down what it was doing, what it hit and the options it can see, then hands that to an engineer as a comment or draft pull request. The engineer answers in a sentence and work resumes. A good stop costs ninety seconds; a bad improvisation costs a weekend.
Who is accountable when the code was generated
The engineer who merges it. That is the whole governance philosophy in one line, and every other control on this page exists to make it fair to ask.
Nothing about the origin of a change moves responsibility for it: not the model, the agent, the vendor, or the prompt author if somebody else approved the merge. Teams that accept this stop arguing about tooling and start reading diffs. Teams that do not eventually get a production incident belonging to nobody, the worst kind to run.
Governance that does not cost you speed
The common failure here is not weak policy. It is policy nobody follows. It fails the same way every time. Somebody who does not write code writes a document. It requires a form, adds a gate with no owner and forbids the tool the team is productive in. Six weeks later everyone has routed around it and you are worse off, holding written evidence of a control you do not operate: a liability in the exact audit it was written for.
Three rules make the difference
Automate the gate rather than asking people to remember it
Anything depending on human memory at six on a Friday fails on a Friday.
Put the rule where the work happens
A rules file the editor loads, a CI check that blocks, a pull request template, not a wiki nobody opens twice.
Give every control an owner and a reason
So when it becomes annoying there is somebody to argue with rather than a document to ignore.
Then size it to the team you have
A team of five
One page of policy, an approved tool list, stop conditions in a rules file every repository loads, four or five CI gates, and a rule that nobody merges their own generated change. No committee, no exception process, because the exception process is asking the other four. Anything heavier is dropped by the second sprint.
A team of fifty
Several squads and probably an enterprise customer. The same controls plus the machinery to survive scale: an owner per control, an exception route with an expiry date, a quarterly refresh of the model list, onboarding that teaches the rules in week one, and evidence collection that runs automatically, because asking fifty engineers for evidence returns forty-one answers of varying accuracy.
What does not change with size: stop conditions, the accountability line, gates that block in CI. Only the process around them changes. We measure the pilot on cycle time, review turnaround and change failure rate. Controls costing more than a few percent of cycle time are the wrong controls.
What you can show when procurement asks
The question now appearing in security questionnaires reads roughly: describe how AI tools are used in your software development lifecycle and how their output is controlled. A governed SDLC lets you answer with artefacts instead of adjectives: a tooling policy with a version and review date, the stop conditions as they exist in the repository, CI configuration showing which gates block a merge, the review rubric, the provenance record, dated evidence of who was trained.
To be precise about what that is worth: it supports readiness for a customer security review, and the practice can be assessed against the control areas a SOC 2 or ISO 27001 process examines, so the work counts twice. It certifies nothing, and we are not a certification body. It turns an uncomfortable meeting into a document you can send. Where the questions turn to the software itself rather than the process, an AI Application Security Audit produces the matching evidence.
How the engagement runs
Assess current practice
week 1
Interviews with the engineers doing the work, repository and pull request history, CI configuration, whatever documentation exists. We ask which tools are in use rather than which are permitted, and the distance between those answers is usually the most useful thing we learn all week. We also read a sample of merged changes to see what review currently catches, which is more honest than any description of it.
Design
week 2
We draft the policy set against how your team already works rather than a template. Every rule gets an owner, a reason and a mechanical check wherever one is possible. Anything that cannot be checked mechanically becomes a rubric line, so it lands on a person deliberately rather than by accident.
Pilot
weeks 3–4
One team, one or two repositories, controls in warning mode. We measure cycle time, review turnaround, how often stop conditions fire and whether they fired for good reason. Rules that fire constantly over nothing are removed here, because they teach a team to ignore the set.
Roll out
weeks 5–6
Extend to the remaining teams, switch CI gates from warning to blocking, commit context and rules files into every repository, run the training session, brief whoever customers will ask, so the answer never depends on one engineer being available.
Review
90 days
We look at what is followed and what is routed around, because both are data. Gates get tuned, the model list refreshed, rules costing more than they return removed rather than defended. A policy written in July is stale by Christmas.
Six weeks for a single team. Multiple teams, several stacks or a regulated environment run longer and are scoped individually. The pilot is measured on cycle time, review turnaround and change failure rate, so a control that is not earning its keep is removed rather than defended.
What you receive
- A written AI development policy set covering approved tooling, data-handling rules for what may be given to a model, prompting standards and the accountability model, versioned and dated so it can go to a customer unedited.
- Agent stop conditions as working configuration, written into the rules and settings files of your team’s tools, so the control lives in the toolchain rather than a document somebody must remember.
- Context and house-pattern files committed into your repositories, carrying architecture decisions, naming conventions and security patterns into every session an engineer opens.
- CI configuration as working pipeline files, covering static analysis, dependency and licence gates, secrets detection across history and a changed-lines coverage floor, tuned to your stack rather than run at defaults.
- A code review rubric telling reviewers what to look for in generated changes, with a diff-size ceiling and an author declaration, short enough that people use it under pressure.
- A provenance specification describing how AI-assisted changes are recorded, where the record lives, how long it is kept and the limits on its use.
- A recorded training session for the engineering team, so new joiners get the same briefing in week one, not an oral history.
- An evidence pack for customer questionnaires, mapping each control to the question procurement asks and the artefact answering it.
How we assess governance maturity
This engagement reports against the Engineering Governance dimension of The Noseberry Product Assurance Framework, on the same evidence base every assessment in the cluster uses: how changes were actually reviewed and merged, what CI enforces, what is written down, and interviews about behaviour rather than intent.
The dimension carries one of three statuses.
One person can release unreviewed generated changes to production, no stop conditions exist, and nothing records what was generated.
Practice exists but depends on individuals remembering it, so it holds while the team is small and attentive and stops the week two people are away.
Change follows a defined path surviving somebody’s holiday, and AI use is governed by rules living in the toolchain rather than a document.
We assess the process, never the people in it. Team structure, performance and hiring are out of scope. This work also complements an AI Code Audit, which reads the codebase you already have while governance changes how the next one gets written.
What changes afterwards
A new engineer is useful in days rather than weeks.
The rules are in the repository and the patterns load into their editor on the first commit, so they need not reverse-engineer four colleagues’ habits to learn what good looks like.
Review becomes judgement again.
Machines raise the findings machines are good at; the human judges whether the change is right.
The questionnaire answer already exists.
When the AI clause arrives, somebody sends the evidence pack that day instead of booking three meetings to invent one.
Incidents get shorter.
Knowing how a change arrived, and who approved it against which rule, removes the first two hours of an investigation.
The drift stops without the speed stopping.
This is what teams are buying. Generation volume stays where it is; it now lands inside boundaries somebody chose, and the next assessment stops finding what the last one found.
Why Noseberry
We run these controls on our own delivery.
The approved model and IDE list, security-explicit prompting, stop conditions, the scan-before-review order: our engineers work under all of it daily on client code. We hand over the policy we live with, adapted to your stack.
We have seen where its absence ends.
The audit, cleanup, security and readiness engagements in this service line put us inside AI-built codebases every week. The findings that recur are the argument for this page, and why these controls are specific rather than aspirational.
Sized to your team, not copied from an enterprise template.
Handing a five-person team a fifty-person policy is the surest way to have neither followed.
One framework across the cluster.
This work reports against the same six dimensions as every other engagement here, so it slots into a wider picture of the product.
Governance is the forward-looking half of the cluster; the Continuous Product Assurance subscription keeps the picture current once the controls are running.
Frequently asked questions
No, and most companies could not enforce it anyway. A ban moves the tool from the company laptop to a personal one; the generated code still arrives in your repository, and you cannot see where it came from or apply a standard to it. An approved list with real data-handling rules gives you the visibility a ban destroys. The exception is one tool with an unacceptable data or licence position, which is a decision about a vendor, not the category.
With artefacts rather than assurances: a versioned tooling policy, the stop conditions as they exist in the repository, CI configuration showing which gates block a merge, the review rubric, a provenance record, and dated evidence that the team was trained. Procurement is not looking for perfection. It wants evidence that somebody has thought about this and that the thinking is enforced somewhere other than a document nobody reads. The evidence pack we deliver maps each artefact to the question it answers.
Properly designed, by a few percent of cycle time, and it usually repays that within a month by removing rework. Badly designed, it slows a team severely, which is why the pilot measures cycle time, review turnaround and change failure rate before and after. Gates that run automatically cost seconds. What costs real time is process requiring somebody to remember, ask permission or fill in a form. We do not use that kind, and controls failing the measurement are removed.
The engineer who merged it. Responsibility follows the merge, not the keystrokes: not the model, not the agent, not the vendor, not the prompt author if somebody else approved the change. That is a fair line to draw precisely because the surrounding controls make it fair: stop conditions prevent an agent deciding things nobody delegated, gates catch what machines catch, and the review rubric tells the reviewer exactly what they are signing off. Ambiguous ownership produces incidents nobody can close.
Usually not. The distinction matters at one boundary, where your data goes and what a vendor may retain or train on, and matters most if you handle regulated data, hold contracts restricting third-party processing, or face a security review that asks. For most teams a vendor model on a business agreement with training disabled, plus clear rules on what may be pasted into it, covers the same risk far more cheaply. Self-hosting does not improve the code a model writes.
It gets assessed and usually adopted under conditions. Removing a tool people are productive in is the fastest way to make the whole policy adversarial, and unsanctioned use is the more dangerous kind because nothing constrains it. We look at the vendor’s data handling, retention and training position, decide what may be given to it, and add it to the approved list with those conditions written down. If it cannot be used safely, we say so and help move the team.
By recording a property of the change rather than watching the person. A commit trailer or pull request field states whether a change was substantially generated and by which tool, completed by the author in the normal workflow. No keystroke logging, no editor telemetry, no per-engineer dashboards. The specification carries explicit limits on use: incident analysis, diligence and licensing, never performance management. That limit is not politeness; the moment the data is used against people it stops being accurate.
Almost never. Existing policies were written for code a person typed, and they assume the author understood every line committed. They rarely say anything about what may be pasted into a model, which tools are permitted, when an agent must stop and escalate, how much diff a reviewer can meaningfully assess, or how generated changes are recorded. We extend the policy you already have rather than replace it, so you keep one document set instead of two that quietly disagree.
The AI Code Audit tells you what is in the codebase you already have. This engagement changes how the next one gets written. They pair naturally and often run in sequence: the audit shows what current practice actually produces, and the governance work removes the cause, so the next assessment finds something different. Teams that buy only remediation tend to buy it twice. Teams that buy only governance never find out what is already sitting in the repository.
Six weeks for a single team: assessment, design, a two-week pilot, roll-out, then a review at ninety days. Multiple teams, several stacks or a regulated environment run longer, scoped individually. It is quote-led rather than priced on this page, because the effort depends almost entirely on how many repositories and toolchains the controls have to land in. A scoping call gives you a fixed scope and price, and we will say if less would do.
Decide how your team builds, before it decides for you
Tell us how many engineers you have, which tools they use and what prompted the question. We will come back within two working days with a scope, a price and a view on which controls would earn their keep first.
Related resources
Go deeper with our thinking
From Spreadsheets to a Scalable Platform – building Canada’s Most Tenant- First Coliving System
Read case study
How to Build a SaaS Product
Read guide
The MVP Development Guide
Read guide
Building a Digital Engineering Roadmap That Aligns With Business Goals
Read article
Engineering IoT Products: Key Challenges and Best Practices
Read articleNot sure where you stand?
Take a free two-minute readiness scorecard built for your industry.