I run pre-sales engineering for an agency, which means I’m the person your evaluation process is supposed to catch. I’ve sat through hundreds of these conversations from the sell side, and I’ll tell you the thing most evaluation guides won’t: any agency that says “we’ll own it all” is kind of lying to themselves. It’s never that clean and easy.
That sentence is the whole framework for evaluating a custom app development agency, really. The rest of this piece is how to test for it.
Deals die on ownership, not technology
Earlier this year I watched a deal stall days from signature. Not over price, not over capability. The client was a large enterprise with an existing mobile partner, a backend team, and an infrastructure team, and we were proposing to come in and add to their existing apps. Late in the process they brought everyone into one room, all their partners across the organization, trying to work out who was going to be responsible for what. Who implements what we build. Who owns which parts of the backend.
There wasn’t clearly defined ownership, and that led to a lot of swirl. It nearly paused the whole process.
Here’s the uncomfortable part. That was partly on us. Ownership is something we should cover earlier in the process, and it’s something agencies don’t typically do in pre-sales. So when you’re evaluating an agency, make them do it. Any agency should come in and say clearly, this is the part we’re owning, and just as clearly, I don’t know your internal mechanisms, I don’t know your security and compliance posture, so this part you need to own. An agency that maps its own boundaries is telling you the truth. An agency that claims the whole map is not.
The buyer-side half of this is bringing the right people early. Most evaluations start with a product team that loves an idea and hasn’t involved the constituents who will actually have to make it work. Bring your backend team, your infrastructure team, your existing partners into the process from the start, even if it’s a fly-on-the-wall situation where they’re not making decisions. They’ll ask questions I can’t anticipate, and every one of those questions answered in week two is an escalation you don’t have in month six. Misalignment across the parties who need to deliver together is a death knell for a project.
The tells in a live conversation
Written proposals get polished. Live conversations leak. A few things I’d treat as red flags if I were sitting on your side of the table.
An estimate that comes back too fast. That usually smells to me. It indicates the agency isn’t putting thought into an approach that works for you, just reusing one that worked for someone else.
No technical person on the call. If an agency shows up without someone who can push on the particulars of your systems, your security, your compliance, they’re not serious about scoping the problem. And bring your own technical person too. It’s easy for a sales engineer to get in a room and talk. It’s much harder when someone on your side is pressure-testing the specifics and looking for where the plan would break down. I say that as the sales engineer in question.
Agreement with everything. If you say “we use X, Y, and Z” and the agency simply says “yeah, we can do that,” be concerned. They’re selling you that they’re going to do the work, but they’re not showing you that they have an opinion and experience doing it. An agency should be opinionated about tooling, about test frameworks, about what it has actually shipped with. “We can do anything” is a tell. No one is an expert in everything, and the honest ones name what they’re not a good fit for.
What you can verify without us in the room
Standard evaluation advice covers references and portfolio review, and both are surfaces we curate. Meetings are where agencies are strongest by design, mine included, so weight the things that don’t depend on how well we present.
Go count the production apps. Not the portfolio page, the app stores. How many shipped apps does the agency actually have, and how often do they get updated? An app that hasn’t shipped an update in a year tells you something about what post-launch support really means there.
Look at the open-source record. Does the agency feed back into the community with quality code that anybody can see and evaluate? Public code is the one work sample that can’t be staged.
Read the engineers, not the sales team. Blog posts, conference talks, how the actual builders talk about their work in public. If the only voice an agency has is its sales voice, that’s the only voice you’ve evaluated.
The team you meet versus the team you get
The most documented failure in vendor selection is the switch. Senior architects show up for discovery, and a different team delivers. When a client asks me how to test for that, my honest answer is that I don’t have a great one. From the outside, you mostly can’t. It becomes contractual. Get the roles and seniority you were shown written into the agreement, with a mechanism that governs substitutions.
Contract structure is itself a tell. An agency that will only work rigid time-and-materials is handing you open-ended risk with no real control mechanism. An agency that will only work a rigid fixed bid is telling you it won’t learn as it goes or flex with your needs. The good middle path fixes outcomes and defines the mechanism for changing scope, because scope always changes. If other vendors own portions of the delivery, write that in too, with an escape mechanism for when their priorities shift.
Then test communication access, because it’s the best live proxy for continuity you have. Ask directly: can we join your standups? Can we share a Slack channel? Can one of our engineers sit in and participate? Our answer is always that the more you can get involved, the better this will go, and an embedded client engineer usually helps us more than it helps you. They know your systems. Any agency that says “just hand us the spec and we’ll give it to you when we’re done” is a red flag, because you don’t know what you’re getting until the very end, and that’s a bad answer.
Watch responsiveness in pre-sales too. A quick reply, even one that says “I don’t know, let me get back to you in 24 hours,” is a good sign, because communication culture flows through the rest of an organization. What you see in the front of the process is what you’ll get in the back of it. I’ve seen the opposite burn projects. A question gets asked, the team scrambles, the answer gets delayed, and trust drops. That’s a harbinger of a project going off the rails. Fast is good when someone is answering you. Fast is bad when someone is pricing you. One takes a communication culture, the other takes actual work.
One more thing to ask for, because most clients don’t: documentation. With AI in the delivery process, decision records and communication logs are table stakes now. Ask how you’ll see them. The answer you want sounds like ours: you’re in our project tracker, you have repository access, you can watch architecture decision records and decision logs land with each change.
What AI actually changes
Every CTO evaluating agencies right now is asking some version of the same question. My own engineers have AI tooling, they can ship features internally, so why hire an agency at all?
They’re partly right, and the honest answer starts there. If you’re looking for a body shop, AI has made that a worse purchase than ever. The reality is that AI makes parts of the process more compressed. It doesn’t collapse it completely. The lifecycle right now is somewhat lumpy, with AI accelerating some stages while the communication points between design, engineering, QA, product, and with client decision makers lagging behind.
AI is a tool at this point. A really good tool. But knowing what you need to do to harden an application or scale one is required to get the tool to do the thing you want it to do, and that knowledge has to span the whole system architecture, frontend, backend, integrations, and data sources. DiVine came to us exactly this way. Their team had vibe-coded a working prototype on their own, and the work we did together was everything between that prototype and a stable, production-ready application. That’s the value an expert agency has left to sell, and it’s substantial: how to do it right the first time, how to plan for it, how to sequence it so it succeeds in production.
Where we’d tell you no
A framework like this should implicate its author, so here’s ours. If you approached us and said build the Flutter app, and also build the backend and all the infrastructure to run everything, that would give me a lot of pause. We know where we’re experts. We’re experts in Flutter. We can do full-stack work on the backend, but crossing into cloud architecture is where you might need an expert partner for that piece.
That’s also the signal to look for in anyone you evaluate. A good agency reads your RFP and asks about the parts it doesn’t cover. Who’s doing the other components? Has this been thought through, or is it in another RFP? Not just answering the mail, but looking at the whole landscape and saying, we can do this part well, and this other part needs someone. Knowing where our limits are, knowing where yours are, and being open when the fit isn’t there.
Be the partner you want your agency to be
I closed a previous piece on RFPs by telling buyers to write the RFP they’d want to receive. The companion rule for evaluation is this: be the partner you want your agency to be. Be clear, communicative, and open about what you’re doing, where you are, and where you’re going. Hiding things or deliberately holding back information is going to cost you in the long run, because every evaluation is really a preview of the working relationship, and you’re half of it.
The agencies worth hiring will show you their boundaries, their engineers, their decision records, and their limits without much prompting. Test for that. And give them the same honesty back, because the projects that succeed in production are the ones where both sides told the truth from the first call.
The tells in this piece are your checklist: mapped ownership boundaries, a real technical voice on the call, public code you can read, a substitution clause in the contract, and genuine communication access. Very Good Ventures is a Flutter consultancy, and we take on custom app development for organizations building multi-platform apps from a single codebase, from mobile to web to embedded. We would rather you run that checklist on us than take our word for it. Talk to our team about your project, even before you have written the RFP.
