Digital Engineering/AI Product Assurance/Vibe Code Cleanup
Fix the AI-built product without taking it offline
You already know the codebase is a problem. You do not need another opinion about it. You need someone to go in and change it, component by component, while customers keep using the product.
What “vibe code” actually is, and why cleaning it up is a discipline
“Vibe coding” describes software written by describing intent to a model and accepting what comes back. It is an effective way to reach a working product, and the code is usually competent line by line. The characteristic problem is not quality. It is memory.
Each generation session knows the prompt in front of it and little about what was decided three weeks earlier. The product accumulates parallel solutions to the same problem, conventions that hold in some directories and not others, and behaviour that lives only in the running system because nobody wrote down what it was meant to be. The software is not badly written. It is unowned.
That distinction determines what remediation has to be. This is not a tidy-up. Renaming variables, deleting dead files and applying a formatter changes nothing about the risk you carry. Cleanup is the work of establishing what the system currently does, deciding what each part of it should become, and moving it there in increments small enough to prove nothing broke.
The discipline sits in three constraints. Behaviour must be preserved unless a change is explicitly agreed, which means capturing it before anything moves. Work must be ordered by risk and dependency rather than by what is annoying, because the annoying part is rarely the dangerous part. And every verdict must be recorded with its reasoning, so your own engineers can later see why a module was left alone rather than assuming it was missed.
What a cleanup covers
Remediation is scoped against the six dimensions of The Noseberry Product Assurance Framework, so work is traceable back to findings and forward to a reassessment.
Code Health
Collapsing duplicated implementations onto one, imposing consistent boundaries and naming, breaking apart files that have become dumping grounds, and building the test coverage that lets the rest of the work happen safely.
Making authorisation consistent rather than incidental, moving secrets into managed configuration, applying input validation at the boundary instead of wherever somebody remembered, closing exposure paths an AI Application Security Audit has located, and resolving dependency and licence conflicts before they become a legal conversation.
Repairing data models shaped for a demo, giving records a single owner, separating responsibilities merged for speed, and replacing integration code that assumes the happy path with code that behaves when the third party does not.
Scalability & Performance
Indexing, query rewriting, pagination, caching where it earns its complexity, and moving work that blocks a request into background processing that can be observed and retried.
Real environments, infrastructure defined in code rather than clicked into a console, a deployment path with a rollback, and alerting good enough that you learn about failure from a monitor rather than a customer.
Engineering Governance
Branching, review and release rules your team can keep after we leave, architecture decisions written down, and explicit conditions for where AI tooling is used and how its output is verified.
Retain / Refactor / Replace: how every component gets its verdict
Cleanup goes wrong when it becomes taste. Retain / Refactor / Replace exists so that each component receives one verdict, produced by a stated test, recorded with its reasoning and its effort. You can disagree with a verdict. You cannot be confused about why it was reached.
It works, it is understood, and change pressure is low.
All three must hold before a component is left alone: it behaves correctly, someone on your side can explain it, and your roadmap has no plans that touch it. This includes the underrated discipline of leaving ugly-but-stable code alone. A module that has run untouched for eleven months is a solved problem with unattractive formatting, and rewriting it converts a stable asset into an untested one. Retained components are recorded as retained, so nobody later assumes they were overlooked.
The logic is correct, the structure is not.
The behaviour is what you want; the shape of the code makes it expensive to change, hard to test, or impossible to reason about. Refactoring keeps the logic and moves it: extracting responsibilities, removing duplication, introducing seams so the thing becomes testable. The test is whether you would keep the current behaviour if you rewrote it from scratch. If yes, restructure under test rather than rewrite.
The cost to understand exceeds the cost to rewrite.
Both halves are required: reverse-engineering the component must take longer than rebuilding it, and its behaviour must be specifiable from outside by observing inputs and outputs, without trusting its internals. Where behaviour cannot be specified from outside, the component gets characterisation tests and a Refactor verdict first, and Replace is reconsidered once its contract is known.
How the verdicts fall in practice. An authorisation layer with checks scattered across routes is usually Replace: the required behaviour is completely specifiable, namely who may do what to which record, and the existing implementation is not worth untangling. A data model is almost always Refactor, executed through expand-and-contract migrations, because the data is real and cannot be discarded even when its shape is wrong. A payment integration is normally Retain for the parts that have processed money correctly, with a targeted Refactor of retry, idempotency and webhook handling, since code that has settled thousands of transactions carries earned reliability. UI components split: a design system is worth Refactoring onto shared primitives, a one-off screen nobody has edited since launch is Retain. Background jobs are frequently Replace: narrow contracts, unreviewed internals, silent failure.
How verdicts become a plan. Replace items carrying security or data-integrity exposure lead, because they remove the most risk per week. Refactors other work depends on come next; the data model usually sits here and gates everything above it. Independent refactors follow, sequenced around your delivery so feature work is not frozen. Retain items produce no work at all, only a documented decision.
Refactor in place, the usual answer
A rebuild discards working business logic and every undocumented lesson inside it, and reliably runs two to three times its estimate while feature delivery stops. In the overwhelming majority of cleanups the problem is structure, ownership and consistency rather than the technology choice, so incremental remediation keeps the running product and pays back sooner.
When a rebuild is genuinely right
- The data model cannot express the product you now sell.
- The platform blocks something you have already sold.
- More than roughly two-thirds of components independently qualify for Replace, at which point incremental replacement has become a rebuild with extra coordination.
The Vibe Code Cleanup Decision Matrix
The full test grid, the evidence each verdict requires and the sequencing rules, as a one-page reference.
How your product stays live while we work
The fear is reasonable and almost always unspoken: that the cleanup is worse than the problem, and the product spends a fortnight broken while strangers rearrange it. Here is the mechanism preventing that.
Behaviour is captured before anything moves.
Before a component is touched, we write characterisation tests recording what it currently does, including the parts that are arguably wrong, because “wrong but depended upon” is a real category. Those tests are the contract. If a change makes one fail, either the change is wrong or it was intended, and the second case is agreed in writing before it ships.
Work happens on short-lived branches off your existing flow.
We do not run a long-lived “cleanup branch” that diverges for six weeks and returns as a merge nobody can review. Each verdict becomes one or more small branches, opened as pull requests against your main line, reviewed by your engineers as well as ours, and merged within days. Your team ships features in parallel.
Releases are incremental and independently reversible.
Every change lands as its own release rather than an accumulated batch, so if something regresses you know precisely which change caused it. Riskier replacements go out behind a feature flag with the old path still present, letting us route a fraction of traffic first, compare behaviour, and switch back instantly without a deployment.
Data changes use expand and contract.
New structures are added alongside the old ones, both are written to while reads migrate, and the old structures are removed only after the new ones have been correct in production for an agreed period. No migration requires a maintenance window or a one-way door.
Rollback is tested, not assumed.
Before the first change, we verify that you can actually reverse a release and that a restore works, because a rollback plan nobody has executed is a hope. There is no big-bang cutover at any stage.
Who this is for
Founders whose product has stopped being changeable.
You built it, it earns money, and it now takes days to do things that should take hours. You are not looking for validation of how it was built. You want the resistance gone.
Teams that have already had the audit.
You hold a findings register, the Critical items are clear, and internal capacity to execute them does not exist without stopping the roadmap. Cleanup is the execution layer for that document, whoever produced it.
Product owners who inherited it.
The person who built the product has gone. You need the system made legible before you can plan anything, without a freeze you cannot justify upstairs.
Engineering leads with a deadline.
A security review, a migration or an enterprise onboarding is scheduled, and the codebase will not survive contact with it.
Signals it is time to clean up rather than carry on
Your engineers have started routing around parts of the codebase.
When a team quietly agrees not to touch the billing module, that module has already failed. The avoidance is the diagnosis.
Fixes generate fixes.
A one-line change breaks something with no obvious relationship to it, twice in a month. That is coupling nobody can see, and it worsens with every feature added.
Nobody can estimate anything.
Estimates have stopped being useful because the surprise per task is now larger than the task. Planning becomes theatre.
The same incident keeps recurring in different clothing.
Related failures sharing a cause that never gets addressed, because each one is fixed as a symptom under time pressure.
A dependency or provider has forced your hand.
Something external is end-of-life or has changed its terms, and the change cannot be absorbed because the integration is spread through the product.
How a cleanup runs
Component inventory and verdicts
We map the system into components at a granularity that can carry a decision, whether a module, a layer, an integration or a job, and give each one a verdict against the tests above, with effort attached. Where a recent audit exists, this stage starts from its findings register rather than repeating it.
Plan and agreement
The verdicts are sequenced into a delivery plan by risk, dependency and your roadmap, and we walk you through it before any code changes. This is the point to argue. Anything you want deferred is deferred, in writing, with the consequence stated.
Safety net
Characterisation tests around the components due to change, a verified rollback path, and whatever observability is missing to tell whether a release behaved. Nothing structural moves until this exists. The stage produces no visible progress and is the reason the rest is safe.
Execution in increments
Small branches, reviewed pull requests, independent releases, feature flags on the risky replacements, expand-and-contract on the data. Your engineers review our changes throughout, which is how the knowledge transfers to the people who will own it.
Handover and reassessment
The system is reassessed against the six dimensions so you can see the movement, and we hand over the architecture documentation, decision record, test suite and release process. Where a governance gap caused the drift, we leave the rules that close it.
Progress is continuous. Verdicts and merged work are visible in your repository as they happen, not summarised at the end.
What you receive
- A component inventory with a recorded verdict for every part of the system, including the parts marked Retain, each carrying the reasoning behind the decision.
- The remediated codebase itself, delivered as reviewed and merged pull requests in your own repository rather than a handover archive, so the history of every change stays with the code.
- A behaviour-preservation test suite covering the components that changed, written to be run by your team on every future release rather than abandoned when we leave.
- A deployment and rollback path you have exercised, with the environments and configuration handling to support it.
- Architecture and data model documentation describing the system as it now is, which for most AI-built products is the first accurate description of it.
- A decision record explaining what was deliberately not done, with the cost of each deferral stated, so future work starts from evidence rather than archaeology.
- A reassessment against the six framework dimensions, so the change in position is measurable rather than asserted.
- A working session with your engineers covering the new structure, the conventions applied and the rules that keep drift from returning.
Source code, and the write access this work requires
An assessment can be done entirely read-only. Remediation cannot, and it would be dishonest to imply otherwise. This is how the access is bounded.
Write access is scoped, named and time-limited.
Engineers are added individually, never through a shared account, to the repositories in scope. Branch protection on your main line stays as it is: our engineers open pull requests and cannot merge them alone. If your repository has no branch protection, establishing it is the first change we make.
Your team holds the merge.
Every change is reviewed on your side before it reaches your main branch, on all changes or only on defined areas as you prefer. What does not work is unreviewed write access, and we decline it if offered.
Production stays yours.
We do not need credentials to live systems, customer records or your production database. Where a migration must run against production data, your team runs it, from our reviewed script, at a time you choose.
Access ends when the work does.
An NDA is signed before scoping. Named accounts are removed on completion and the removal confirmed in writing, and working copies of your source are deleted at close-out unless you ask us to retain them.
How cleanup engagements are shaped and quoted
Cleanup is quoted, not priced from a list, because effort varies by more than an order of magnitude depending on what the inventory finds. What is fixed is the shape of the conversation.
Targeted remediation.
A defined set of findings, usually the Critical and High items blocking a specific event such as a security review or an enterprise onboarding. Fixed scope, fixed fee, a defined completion condition, and the smallest engagement we run.
Full cleanup.
The whole system inventoried and every verdict executed. Quoted in phases against the sequenced plan, each phase individually approved, so you can stop after any phase with working software and no half-finished migration.
Embedded engineering.
Our engineers work inside your team monthly where cleanup runs alongside continuous delivery and priorities move. Suits teams that have engineers and need capacity plus method rather than an outside project.
Every quote follows the inventory stage rather than preceding it, and the inventory is itself a small chargeable piece of work, credited against the engagement if you proceed. You approve a number against a plan you have read.
What changes afterwards
Estimates start meaning something again.
When a change is contained rather than spread, the difference between a two-day task and a two-week one is visible before you commit. Planning stops being a negotiation with an unknown.
Deployment becomes uneventful.
A release that can be reversed in a minute is a release nobody schedules a meeting about. Teams typically ship more often within weeks, because the cost of each release dropped.
The security questionnaire has answers.
Consistent authorisation, managed secrets and a documented release path convert most of a procurement questionnaire from “I don’t know” into a factual answer with evidence behind it.
A second engineer becomes useful quickly.
Documented structure and a test suite people trust replace the onboarding method of reading the entire system.
You stop paying interest.
The tax AI-built software charges is not the code, it is the time spent re-deriving what the code does before every change. That is the cost cleanup removes, and it is paid back weekly.
Why Noseberry for this work
We remediate systems we did not write, routinely.
Cleanup on somebody else’s code is a distinct skill from building new software: reading unfamiliar systems, inferring intent, changing them without breaking the parts nobody documented.
The verdicts are tested, not opinionated.
Retain / Refactor / Replace is published, including the conditions under which we recommend leaving code alone and those under which a rebuild is genuinely correct. A supplier whose framework always concludes “rewrite it” is describing their revenue model rather than your system.
We use AI tooling under governed conditions and say so.
Approved models and IDEs, security-explicit prompting, automated scanning before human review, and explicit stop conditions so an agent escalates rather than improvises. The same rules apply to code we generate during remediation.
Your team reviews everything.
Knowledge transfer is not a closing presentation, it is your engineers reviewing our pull requests throughout. When we leave, the people who own the software have already read every change.
One framework across the engagement.
The six dimensions used to assess the system are used to measure it afterwards, so improvement is comparable rather than claimed.
Frequently asked questions
Fix it, in almost every case, but decide component by component rather than as one verdict over the whole product. A rebuild discards working business logic and the undocumented lessons inside it, and it reliably overruns while feature delivery stops. Rebuilding is genuinely correct when the data model cannot express the product you now sell, when the platform blocks something you have already promised a customer, or when more than two-thirds of components independently qualify for Replace. Our inventory answers that with evidence in days.
No planned downtime, and the method is built around that constraint rather than hoping for it. Changes land as small, independently reversible releases rather than one large cutover. Riskier replacements run behind feature flags with the previous path still in place, so traffic can be switched back instantly without a deployment. Data changes use expand-and-contract migrations, which never require a maintenance window. Before the first change we verify your rollback and restore actually work, because an untested rollback is not a rollback.
Targeted remediation of a defined set of Critical and High findings typically runs two to four weeks. A full cleanup of a single-product codebase is usually six to twelve weeks, phased, with working software and a stopping point at the end of each phase. Multi-repository estates and systems with heavy integration surfaces are scoped individually. The inventory stage gives you a phased plan with durations attached before you commit to the whole programme, so the timeline is agreed rather than discovered.
Not necessarily. Cleanup begins with its own inventory stage, which produces the verdicts and the plan, so you can start here directly. An audit is worth doing first when you need a diagnosis for someone else, a board, an investor, an enterprise customer, or when you have not decided whether to invest at all. If you have already had an AI Code Audit with us or anyone else, we start from that findings register rather than charging you to rediscover it.
It is quoted against scope rather than published, because the same brief can mean three weeks or three months depending on what the inventory finds. Quoting before that would be a guess dressed as a price. The inventory is a small, chargeable piece of work that produces a phased plan with effort attached, and its cost is credited against the engagement if you proceed. A fixed entry price exists for diagnosis on the AI Code Audit page; remediation is deliberately not sold that way.
Yes, and this is the main practical difference from an audit. Named engineers are added individually to the repositories in scope, never through shared credentials. Your branch protection stays in force: we open pull requests and your team merges them. We do not need production credentials, customer data or database access, and where a migration must run against production your engineers run our reviewed script at a time you choose. Access is removed at completion and the removal confirmed in writing.
Yes, and most clients do, because a delivery freeze is rarely affordable. Work runs on short-lived branches against your existing flow rather than a long-lived cleanup branch that diverges for weeks and returns as an unreviewable merge. Where a refactor touches an area your team is actively building in, we sequence around it or pair on it rather than compete for the same files. The plan is built against your roadmap, and any conflict between remediation and delivery is raised as a scheduling decision for you.
Almost never, and never as a default position. Changing platform is a Replace verdict applied to the entire system, which carries every risk of a rebuild plus a hiring and operations change. We recommend it only where the current platform blocks a commitment you have already made, and we will say so explicitly with the cost attached. The overwhelming majority of cleanups leave the stack exactly as it is, because the problem is structure, ownership and consistency rather than the technology choice.
Characterisation tests are written before a component is touched, recording what it does today, including behaviour that is arguably wrong but which customers may depend on. Those tests become the contract for the change, and they run on every release rather than once at the end. A failure means either the change was incorrect or it was intended, and intended behaviour changes are agreed in writing before they ship. You get that suite as a deliverable, so the same protection covers work your own team does afterwards.
Cleanup that leaves no rules behind is a temporary purchase. Handover includes branching, review and release conventions your team can sustain, architecture decisions written down, and explicit guidance for where AI tooling is used and how its output is verified before merge. Where the original drift came from a governance gap rather than a coding one, that gap is named. Teams wanting the position checked continuously can move to Continuous Product Assurance rather than repeating this engagement in a year.
Get the verdicts before you commit to anything
Send us the repository and a short note on what has gone wrong. We come back within two working days with a scope for the inventory stage, after which you hold a verdict on every component and a phased plan with effort attached, before approving a single line of remediation.
Related resources
Go deeper with our thinking
From Spreadsheets to a Scalable Platform – building Canada’s Most Tenant- First Coliving System
Read case study
How to Build a SaaS Product
Read guide
The MVP Development Guide
Read guide
Building a Digital Engineering Roadmap That Aligns With Business Goals
Read article
Engineering IoT Products: Key Challenges and Best Practices
Read articleNot sure where you stand?
Take a free two-minute readiness scorecard built for your industry.