How to Build the Internal AI Team at a Mid-Market Financial Institution

The bank hired three data scientists. They were excellent — strong Python, good intuitions about modeling problems, one had a PhD. They spent eight months building models. At the end of those eight months, none of those models had reached production. Not because the models were bad. Because nobody was leading the program work: the data wasn't clean enough and nobody was accountable for fixing it, the business lines didn't understand what was being built for them, Model Risk had never been brought into the conversation, and there was no production infrastructure to deploy into. The data scientists had done their jobs. The job that nobody was doing was the one that would have made their work matter.

This is the most common failure mode I see in AI program buildouts at mid-market financial institutions. The sequence is wrong. Technical capability gets hired before the leadership and organizational infrastructure that makes technical capability productive. The result is good work that goes nowhere — and eventually, talented people who leave because they are frustrated by the same friction every quarter.

The right sequence is not complicated, but it is counterintuitive enough that most institutions get it wrong the first time.

Why mid-market banks hire backwards

The logic is understandable. An institution decides to build an AI program. The visible output of an AI program is models. Models are built by data scientists. Therefore, hire data scientists.

This is wrong for a reason that becomes obvious in retrospect: models are not the output of an AI program. Production outcomes are the output. A model that is not in production — that has not cleared governance, has not been integrated into the workflow that uses it, has not changed how a decision gets made — has produced zero value. The gap between a model that exists and a model that is used is not a technical gap. It is an organizational and program leadership gap.

Mid-market institutions are particularly susceptible to this mistake because they are often looking at large banks for a reference model. Large banks have dozens of data scientists. They have AI labs. The AI program looks, from the outside, like a technical organization. What is invisible from the outside is the program management infrastructure, the executive sponsorship, the dedicated data engineering teams, the model risk management integration points that were built over years — all the organizational plumbing that makes the technical work move. A mid-market bank that hires three data scientists and calls it an AI program has the most visible component and none of the supporting structure.

The four roles an AI team at a mid-market bank actually needs

I've structured AI teams at institutions ranging from $3 billion to $40 billion in assets, and the team shape that works is consistent. Four roles. The sequencing of when you hire or contract each one is as important as the roles themselves.

An AI program lead. This is the role to fill first, and it is not a data scientist. It is a program leader with AI fluency — someone who can identify which problems in the institution AI can actually solve, prioritize them against the institution's constraints and risk appetite, define the build plan, drive the organizational work across IT and Risk and the business lines, manage vendors and implementation partners, and report to the executive sponsor in a way that keeps them engaged without drowning them in technical detail.

AI fluency means they understand how the technology works well enough to have credible conversations with the data science team and with vendors — not building models themselves, but knowing when a proposed approach is realistic and when someone is overselling capability. Financial services experience is non-negotiable. The specific blockers in an AI program at a bank — MRM, data governance, examiner expectations, fair lending review — are specific to this industry. A program lead who has navigated these at another institution is worth considerably more than one who hasn't.

This is also the role where a fractional arrangement makes the most sense in the early stages. I'll come back to that.

A data engineer. This is the most underappreciated dependency in a bank AI program, and it is almost always under-resourced. Data scientists build models. Data engineers build and maintain the pipelines that get clean, structured data to those models — at training time, at validation time, and in production. Without a data engineer, data scientists spend 60% to 70% of their time doing data preparation work that is not their highest use, the data pipelines are brittle and manually maintained, and getting a model into production requires IT involvement at every step because there is no team infrastructure to deploy into.

At a mid-market bank, data is particularly messy — core banking systems that have been in place for decades, data sitting in formats that were not designed for ML consumption, regulatory requirements about data lineage and provenance that have to be satisfied before a model can be validated. A data engineer who has worked in financial services and understands both the technical and the regulatory constraints around data is genuinely hard to find. This is the role that most institutions either skip, understaff, or try to cover with a generalist who ends up stretched too thin.

A model builder. This is the role everyone hires first. At a mid-market institution with a focused use case portfolio, one strong model builder can carry a significant amount of work — particularly if the program lead has done the job of defining clear problem statements, the data engineer has built reliable pipelines, and the governance infrastructure is in place so that models can move through review without months of rework. The mistake is not hiring model builders; it is hiring them before the infrastructure that makes their work productive exists.

At the mid-market level, "model builder" often means someone who can work across a range of techniques — classical ML for structured credit and fraud problems, LLM-based approaches for document processing and knowledge retrieval — rather than a deep specialist in a single method. Breadth is more valuable than depth at this team size.

An AI risk and governance lead. Regulators are increasingly explicit about this role, and the institutions that have tried to fold it into existing risk or compliance functions have mostly found it doesn't work. The reason is that AI governance requires fluency in how AI systems actually work — the specific failure modes, the monitoring requirements, the documentation standards for MRM review — combined with the regulatory knowledge of what examiners are looking for. A traditional credit risk manager can learn the regulatory side quickly but often lacks the AI-specific technical understanding. A data scientist can build models but is usually not the right person to own the governance relationship with MRM, Legal, and Compliance.

At a mid-market institution, this role may start part-time — one person spending 30% of their time on AI governance alongside other risk responsibilities — and grow to full-time as the model inventory grows. The important thing is that it is a named role with a named owner, not a set of tasks that everyone assumes someone else is handling.

Hire, contract, or fractional — and when

The sequencing question is when to hire full-time, when to use contractors, and when a fractional arrangement makes sense. The answer depends on how mature the program is and what the institution's current bandwidth is.

In the first twelve to eighteen months of an AI program, the work is highly variable and the institution is still learning what it actually needs. This is a poor environment for a full-time team build. The first use cases haven't reached production yet, which means the team's workload is not predictable, and the institution doesn't yet know whether it needs two model builders or five, whether the data engineering work is best done internally or by a partner, or what the right permanent structure looks like.

This is the phase where a fractional program lead — someone who can run the program, make the organizational decisions, and define what the permanent team should look like — combined with targeted contractor or staff augmentation for specific technical needs, is usually the most efficient structure. The fractional program lead is doing the work while simultaneously building the internal understanding that will allow the institution to make better full-time hiring decisions when the program is mature enough to justify them.

Contractors work well for data engineering and model development work with defined scope. They work poorly for governance and for anything that requires sustained institutional knowledge — relationships with business line owners, history of past decisions, an understanding of the data landscape that takes months to accumulate.

Full-time hires make sense once the program has enough stable, ongoing work to occupy a role consistently, and once the institution is clear enough about the role definition to hire against it accurately. Hiring a full-time AI governance lead before the institution has a clear picture of what AI governance looks like in its specific regulatory context often produces a mismatch between what the person was hired to do and what the job actually turns out to be.

What to look for in each role — and the credentials that don't predict success

For the program lead, the credential that matters most is a track record of getting AI or technology programs to production outcomes inside a regulated institution — not a list of certifications, not a PhD, not a title at a large bank that doesn't tell you what they actually did. Ask for specific programs they ran, what reached production, what didn't and why, and what they would do differently. The answers will tell you more than the resume.

The credential that consistently fails to predict success in a program lead role is a data science background without program leadership experience. Excellent data scientists who move into program leadership often struggle because the job is fundamentally different: it is about driving organizational decisions across functions that have competing priorities, not about solving technical problems with defined inputs and outputs. Some people make the transition successfully. Most find it more difficult than they expected, and the program pays the cost during the learning period.

For the data engineer, financial services data experience matters more than pedigree. Someone who has worked in the data infrastructure of another financial institution — who has seen core banking data, who understands what a data lineage requirement looks like in practice, who has built pipelines under regulatory constraints — will be productive faster than a strong engineer with no regulated-industry experience.

For the model builder, watch for people who conflate model performance on a benchmark with production viability. The relevant skill is not building a model with the highest accuracy on a holdout set; it is building a model that can be validated, explained to a non-technical governance committee, monitored in production, and maintained when it drifts. These require different instincts than competitive ML work, and the interview process should test for them explicitly. Ask how they've handled model governance reviews, what their experience is with model documentation for MRM, and what monitoring they built into the last model they put into production.

For the governance lead, regulatory knowledge plus AI fluency is the combination. It is uncommon, which is why the market for credible AI governance professionals in financial services is tight. Someone who is strong on one dimension and weak on the other can be developed, but the development timeline is long enough that it is worth being deliberate about which dimension you are hiring for and which you are developing.

How the team relates to IT, Risk, and the business units

The AI team is not a technology team. It is not a risk team. It is not owned by any single business unit. Getting this structural question right early prevents a significant amount of friction later.

The team should report to a senior executive with cross-functional authority — the COO, the Chief Strategy Officer, or in some cases the CEO directly. Reporting into IT produces a team that is perceived as a technology function and loses credibility with business lines that don't want IT telling them how to run their operations. Reporting into a single business line creates a team that serves that line effectively and is resented by every other line as playing favorites.

The relationship with IT is a partnership, not a reporting line. The AI team defines what it needs in terms of data infrastructure, compute, and deployment environment. IT builds and maintains it. The specific handoff points — who owns the production deployment, who owns the data pipelines in steady state, who is on call when a model fails at 2am — need to be written down and agreed to before they become a problem.

The relationship with Risk is similar. Model Risk Management is a governance function, not an adversary. The AI program lead should be meeting with the head of MRM on a regular cadence — not just at validation submission time — so that the governance relationship is collaborative rather than adversarial. The use case prioritization process should include a risk assessment that MRM has input into, so that models are not arriving at MRM for the first time at validation.

Business line relationships are the most important and the most frequently neglected. The AI team needs a named sponsor in each business line it is building for — someone at the VP or SVP level who owns the use case, is accountable for defining the requirements, will champion the change management, and will be the honest signal about whether the model is actually useful in production. Without that sponsor, the AI team is guessing about requirements, and the model it delivers will be technically correct and practically unused.

What the team should look like in three years

The three-year picture at a mid-market institution with an active program typically has the following shape: a permanent AI program lead or Chief AI Officer who is a full-time internal hire, a data engineering function that is either internal or a well-managed external partner with deep institutional knowledge, two to four model builders depending on the use case portfolio, a dedicated governance function that owns the MRM relationship, model monitoring, and examiner preparation, and embedded AI leads in the two or three business lines where AI is most active — people who sit in the business but are fluent in AI and serve as the translation layer between what the business needs and what the AI team can deliver.

The fractional or contracted leadership that ran the program in year one should have transferred enough institutional knowledge that it is no longer load-bearing. That is the right measure of whether the early-stage structure worked: not just that the program is running, but that it is running under internal ownership that doesn't need external support to function.

The measure of a well-built AI team is not how many models it has built. It is how many models are in production, being monitored, and actually changing decisions — and whether the team that built them can sustain and extend that work without outside help.

If you are an institution that is at the beginning of this build — or that has been at it for a year and is recognizing the pattern described here — the most useful next step is usually a direct conversation about the specific situation rather than a framework applied generically. The constraints are different at a $5 billion institution than at a $25 billion one, and the right sequencing depends on what is already in place.