Why Machine Learning Projects Need Product Managers and Domain Experts

Section 1: Why Technical Teams Cannot Define the Entire ML Problem

Machine learning projects are often viewed primarily as technical initiatives. Data scientists select algorithms, ML engineers build pipelines, and software engineers integrate models into products. But a technically strong model can still fail if the team solves the wrong problem.

This is one of the reasons product managers and domain experts are essential to machine learning projects.

A machine learning system does not operate in isolation. It exists to support a user, improve a business process, reduce a risk, automate a decision, or create some other measurable outcome. Understanding that context requires knowledge that may not be contained in the training data or technical specifications.

Product managers help define what problem is worth solving. Domain experts help explain how the real-world process actually works.

Together with data scientists and ML engineers, they help transform an ambiguous business objective into a model that can deliver practical value.

The Difference Between a Business Problem and an ML Problem

A business problem is usually broader than the machine learning task eventually created to address it.

Consider a company that says, "We need an ML model to reduce customer churn."

That statement does not yet define the actual ML problem.

The organization might need to identify customers who are likely to leave, determine which customers should receive retention offers, identify the intervention most likely to work, or estimate the expected value of retaining each customer.

Each of these is a different problem.

The business objective may be to increase customer retention and lifetime value. The ML task might be classification, ranking, forecasting, or some combination of models.

This distinction is important because technical teams can optimize exactly the wrong target if the original business problem is poorly defined.

A product manager helps clarify the user need, business objective, decision, constraints, and expected outcome before model development begins.

Domain experts add another layer by explaining what the process looks like in practice. A customer-success specialist may know that certain cancellation signals are administrative artifacts rather than genuine indicators of churn. A fraud investigator may know that a particular transaction pattern is normal for a specific customer segment even though it appears unusual statistically.

This information can fundamentally change how the ML problem should be formulated.

The strongest ML teams therefore begin with:

Business problem → Decision → Outcome → ML formulation

rather than:

Business problem → Algorithm

Why Requirements Are Harder in ML

Traditional software requirements often specify what a system should do. A machine learning system introduces uncertainty because its behavior depends on data and learned patterns.

This makes requirements more difficult to define.

A product manager may want "highly accurate fraud detection," but accuracy alone does not define an operationally useful system. The business may care much more about minimizing financial loss while keeping false positives below a level investigators can handle.

Similarly, a recommendation product may initially ask for "the most relevant recommendations." But relevance may need to be balanced with diversity, freshness, inventory, revenue, or long-term customer satisfaction.

ML systems also operate under uncertain data. Teams may discover that important labels are incomplete, historical decisions were inconsistent, or a supposedly useful feature is unavailable at prediction time.

The objective may also evolve as the organization learns more. During experimentation, the team may discover that the original target is not sufficiently predictive or does not correspond closely to the desired business outcome.

This means ML requirements should be treated as an iterative product-development process rather than a fixed specification.

Product managers can help prioritize what matters most as new evidence emerges. Domain experts can determine whether observed patterns make sense within the real-world process.

This collaboration prevents technical teams from interpreting every modeling challenge as a purely algorithmic problem.

The Role of Domain Experts

Domain experts contribute something different: deep knowledge of the environment represented by the data.

They understand how processes work, which events are normal, how outcomes are actually determined, and where edge cases occur.

This knowledge is particularly valuable when defining ground truth.

Suppose a model is being trained to identify fraudulent activity. A domain expert may know that some transactions flagged as suspicious are actually routine behavior for specific customer types. Without that context, the training labels may encode misleading assumptions.

Domain experts can also identify edge cases that are rare in historical data but highly important operationally.

A manufacturing engineer may know that a particular sensor pattern indicates an impending failure even though it appears only occasionally. A security specialist may recognize a new attack pattern that a model interprets simply as unusual network activity.

Experts can also evaluate whether model outputs make sense operationally. A model may be statistically impressive but produce recommendations that cannot realistically be acted upon.

This makes domain expertise valuable throughout the lifecycle—not just during initial data labeling.

Product managers define what matters. Domain experts explain what is true in the real environment. Data scientists and ML engineers determine how to model and operationalize it.

The strongest results emerge when all three perspectives are combined.

"From Prediction to Decision: How Machine Learning Creates Business Value" provides relevant context because an ML prediction creates value only when it improves a meaningful decision and ultimately contributes to a measurable business outcome.

Key Takeaway

Technical teams cannot reliably define an ML project in isolation because the hardest questions often concern business objectives, user needs, operational reality, and domain-specific meaning. Product managers help define the problem, priorities, success metrics, and product workflow, while domain experts contribute knowledge about labels, edge cases, processes, and real-world consequences. Their involvement ensures that ML teams do not simply build accurate models, they build models that solve the right problem and can be used effectively in the real world.

Section 2: How Product Managers and Domain Experts Improve ML Quality

Once an ML team has identified a promising business problem, the next challenge is ensuring that the model learns the right patterns from the right data. This is where product managers and domain experts can materially improve the quality of an ML system.

Data scientists and ML engineers understand modeling techniques, evaluation methods, and infrastructure. But they may not know which outcome truly matters to the business, how a label is generated operationally, why certain observations are unusual, or which edge cases have significant consequences.

Product managers provide the connection to the product objective and user experience. Domain experts provide the connection to the real-world process and meaning of the data.

Defining the Right Target

One of the most important decisions in an ML project is defining the target variable.

A team may have plenty of historical data and still train the wrong model if the target does not accurately represent the business outcome.

Consider customer churn. A company might define churn as "account cancellation." That sounds straightforward, but the actual business objective could be reducing customers who are likely to leave and could realistically be retained. Those are not necessarily the same population.

The product manager can help clarify which outcome matters commercially. The domain expert can explain how cancellation occurs in the real process and identify cases where an apparent cancellation does not represent genuine customer loss.

This becomes especially important when teams use proxy metrics.

Sometimes the desired outcome is difficult or expensive to measure directly, so teams use an easier proxy. A recommendation system might optimize clicks because clicks are readily available, even though the business ultimately cares about customer value or long-term retention.

A proxy can be useful, but it can also create objective misalignment. The model may become very good at optimizing the proxy without improving the actual business objective.

Product managers can challenge whether the target metric reflects what users and the business genuinely value.

Domain experts can challenge whether the target accurately represents the underlying phenomenon.

This collaboration helps prevent a common ML failure: building an excellent model for the wrong target.

Improving Data and Feature Quality

Product managers and domain experts can also improve the quality of the data used to train the model.

A dataset contains numbers and categories, but those values have real-world meaning that may not be obvious from their names.

A domain expert may know that a particular field is recorded only after a customer has already entered a certain workflow. That makes the feature unusable for an earlier prediction task because it introduces temporal leakage.

Another field may appear inconsistent because the business process intentionally treats different customer segments differently.

Without domain knowledge, engineers may "clean" such information and accidentally remove valuable signal.

Domain-informed features can also reveal patterns that generic statistical analysis may miss.

A fraud investigator may know that repeated activity across a specific combination of devices, accounts, and locations is suspicious. An engineer can use that knowledge to construct features representing the relevant behavior.

A manufacturing specialist may know that the relationship between temperature and vibration matters only under particular operating conditions. That knowledge can guide the creation of features that better represent the physical process.

Label validation is another major contribution.

A model can only learn what the labels tell it. If historical decisions were inconsistent, labels may contain systematic errors.

Domain experts can perform label validation by reviewing samples and identifying cases where the assigned outcome does not reflect the actual business definition.

Product managers can also determine whether labels correspond to the intended product outcome.

This makes collaboration essential to building trustworthy training data.

Connecting Predictions to Real Decisions

A model's prediction becomes useful only when it can support a real decision or action.

This is where product managers and domain experts provide critical operational context.

Suppose a model predicts that a customer is at high risk of churn. What happens next?

Does a customer-success representative contact the customer? Is an offer automatically generated? Is the case added to a review queue? Does the business have enough employees to act on all high-risk predictions?

These questions determine whether the model is actually useful.

Product managers can design the user workflow around the prediction. The model output may need to appear inside an existing application, be presented as a ranked queue, or trigger an automated action.

Domain experts can determine whether the recommendation is actionable in practice.

A model might correctly identify a high-risk equipment condition, for example, but the maintenance team may not have the parts, staff, or downtime required to act immediately.

The prediction may therefore be technically correct but operationally ineffective.

This is why actionability should be evaluated during model development rather than after deployment.

Teams should understand what decisions the model supports, who makes those decisions, what information they need, and what constraints they face.

Product managers help optimize the complete user experience. Domain experts help ensure that the workflow matches operational reality.

Together, they bridge the gap between prediction and execution.

The same principle applies to human-in-the-loop systems. A model may rank cases for review, but the human reviewers must have enough contextual information and clear escalation rules to make the final decision effectively.

This transforms the ML project from a model-building exercise into a real decision system.

"The Journey of a Dataset: From Raw Data to Production ML" reinforces this broader lifecycle perspective: reliable ML depends not only on model training but also on how data is defined, transformed, validated, connected to production features, and continuously updated.

Key Takeaway

Product managers and domain experts improve ML quality by helping teams define the right target, validate labels, identify meaningful features, uncover edge cases, understand failure modes, and connect predictions to actionable decisions. Their contribution prevents technical teams from optimizing misleading objectives or building models around incomplete interpretations of the real-world process. The result is an ML system that is not only statistically stronger but also better aligned with how the product and business actually operate.

Section 3: Making Cross-Functional ML Teams Work

Machine learning projects become significantly more difficult when multiple disciplines must work together. Data scientists think about model performance, ML engineers think about infrastructure and deployment, software engineers think about application reliability, product managers think about users and business outcomes, and domain experts think about how the real-world process actually behaves.

These perspectives are all necessary, but they can also create friction.

A model may be technically excellent but impossible to integrate into the product. A product requirement may be commercially attractive but impossible to satisfy with available data. A domain expert may reject a model because its predictions conflict with operational reality, while an ML engineer may believe the model is statistically sound.

Successful organizations therefore need shared ownership, effective communication, explicit tradeoff management, and a clear process for moving from experimentation to production.

Shared Ownership

A successful ML product should not be considered solely the responsibility of the data science team.

Product teams are responsible for understanding the user problem, defining business objectives, prioritizing requirements, and measuring product impact.

Data scientists typically focus on problem formulation, experimentation, statistical analysis, and model development.

ML engineers translate models into production systems, build training and inference pipelines, manage deployment, and ensure operational reliability.

Domain specialists contribute knowledge about processes, terminology, edge cases, labels, and real-world consequences.

Shared ownership means that these roles participate throughout the lifecycle rather than appearing only at specific project stages.

A domain expert should not be invited only after a model has already been built. By that point, the team may have invested heavily in a target definition or feature set based on incorrect assumptions.

Likewise, product managers should not enter the process only when the model is ready for launch. Product constraints may fundamentally influence how the model should be designed.

This creates a collaborative lifecycle:

Problem definition → Data → Modeling → Validation → Product integration → Deployment → Monitoring

Each discipline contributes at every relevant stage.

Clear ownership is still necessary. Shared responsibility does not mean that everyone owns everything. The organization should define who makes decisions about the product objective, model quality, data quality, production reliability, and final launch.

Communicating Across Disciplines

Cross-functional ML teams can struggle because different roles use different terminology and optimize different objectives.

An ML engineer may describe a model using precision, recall, calibration, latency, and throughput. A product manager may focus on conversion, retention, revenue, and customer satisfaction. A domain expert may describe the same system in terms of workflow stages, exceptions, policies, and operational risks.

The team needs a shared vocabulary that connects these perspectives.

For example, instead of saying that "recall improved by 4 percentage points," the team should also explain what that means operationally.

Does the increase identify more genuine cases? How many additional alerts will employees receive? What will it cost to investigate them? Does the resulting improvement reduce risk enough to justify that workload?

Translating technical metrics into business implications helps stakeholders make informed decisions.

At the same time, business requirements need to be translated into technically measurable objectives. Saying "make recommendations more relevant" is insufficient for an ML engineering team. The requirement may need to become measurable through ranking metrics, engagement outcomes, conversion, or another clearly defined criterion.

Decision records can help preserve why important choices were made. Teams can document why a particular target was chosen, why certain features were excluded, why a threshold was selected, or why human review was retained.

This becomes valuable later when the team revisits the model or investigates unexpected behavior.

Good communication therefore does not mean removing technical language. It means ensuring that each discipline understands how its decisions affect the others.

Managing ML Projects From Prototype to Production

The transition from prototype to production is where cross-functional collaboration becomes particularly important.

During experimentation, data scientists may focus on maximizing model performance. But production requirements can introduce constraints that were not relevant during development.

The model may need a feature that cannot be computed in real time. The product may require latency below a particular threshold. The operations team may not have capacity to investigate every prediction.

This is why validation needs to extend beyond the model.

The team should validate the data pipeline, feature availability, model latency, product integration, user workflow, and business outcome measurement.

Before deployment, the group should also establish how the model will be monitored. Who will investigate drift? Who owns data-quality alerts? Who decides whether a model should be retrained? What happens when production performance declines?

These questions require participation across roles.

A product manager may own the business success metric. A domain expert may determine whether unusual predictions are plausible. An ML engineer may own the model-serving system. A data engineer may own the upstream data pipeline.

A successful transition therefore requires coordinated decision-making rather than a simple handoff from data science to engineering.

The hidden engineering work behind this transition is substantial. "The Hidden Engineering Work Behind Every Successful Machine Learning Product" explores how data infrastructure, deployment, monitoring, reliability, governance, and continuous improvement determine whether a model becomes a successful product.

Key Takeaway

Cross-functional ML teams work best when product managers, domain experts, data scientists, ML engineers, and software engineers share responsibility for the complete ML lifecycle while maintaining clear ownership of individual decisions. Strong communication connects technical metrics to business outcomes, while structured tradeoff discussions address accuracy, latency, cost, automation, and risk. The goal is to prevent the common handoff problem in which a technically successful model reaches production only to discover that it cannot solve the real business problem or fit the operational workflow.

Section 4: Building ML Products That Reflect Real-World Expertise

The strongest machine learning products are not built by technical teams working in isolation. They are built by organizations that combine machine learning expertise, product judgment, domain knowledge, and operational experience throughout the lifecycle.

A model may identify a statistical pattern, but domain experts can determine whether that pattern makes sense. A product manager may identify a valuable opportunity, but ML engineers can determine whether it is technically and economically feasible. A model may perform well offline, but the product team may discover that users do not trust or act on its predictions.

This makes cross-functional collaboration particularly important after a model reaches production. The system must continue learning from human feedback, operate within appropriate governance boundaries, and adapt as the business environment changes.

Human-in-the-Loop ML

Many ML systems benefit from keeping human expertise inside the decision process.

A model can classify, rank, predict, or prioritize cases, while humans handle decisions that require judgment or involve higher levels of uncertainty.

For example, a fraud model can rank transactions according to risk, while investigators review the highest-priority cases. A healthcare model can identify potentially concerning cases, while clinicians make the final assessment. A customer-support model can classify incoming requests while agents handle unusual or sensitive situations.

This creates a human-in-the-loop architecture.

Expert review generates additional value beyond the immediate decision. Humans can provide feedback that improves labels, exposes model weaknesses, and identifies new patterns that were not represented during training.

Escalation is particularly useful. Instead of forcing the model to make every decision, uncertain or high-impact cases can be routed to specialists.

This also creates a mechanism for continuous learning. Human decisions can become new training examples, provided the organization validates those labels and understands potential selection bias.

Human oversight should therefore be designed as part of the product rather than treated as a temporary fallback for model failures.

Governance and Responsible Product Decisions

ML systems can influence customers, employees, financial outcomes, access to services, and other important decisions. Product teams and domain experts help ensure that those decisions are made within appropriate governance boundaries.

Explainability can become important when users or internal stakeholders need to understand why a model produced a particular prediction.

Risk must also be evaluated in context. A prediction error in a low-impact recommendation system may have very different consequences from an error in fraud detection, hiring, healthcare, or financial decision-making.

Domain experts can identify risks that may not be obvious from aggregate model metrics.

For example, a model may demonstrate strong average performance while failing on a rare but operationally important edge case. A domain expert may recognize that this edge case has significantly greater consequences than its frequency suggests.

Bias is another area where cross-functional review matters. Statistical patterns in historical data may reflect existing business processes or unequal exposure rather than desirable relationships.

Product managers, domain specialists, and ML teams should therefore examine not only whether the model performs well but what behavior the product is creating.

Accountability should be explicit. Teams should know who owns the model, who can approve changes, who monitors production impact, and who is responsible when the system behaves unexpectedly.

Creating a Product Culture Around ML

Long-term ML success requires an organizational culture that treats experimentation, evaluation, and improvement as continuous activities.

Continuous evaluation should measure both model behavior and business outcomes. Teams should know whether the model continues to improve the decision it was designed to support.

Customer feedback can provide an additional source of evidence. Users may identify incorrect recommendations, missing functionality, confusing explanations, or unintended behavior that is not visible through model metrics.

Product managers can translate this feedback into product requirements, while domain experts can determine whether the observed behavior reflects a genuine issue.

The organization should also establish clear long-term ownership. ML products require maintenance just like other production software, but their behavior can change as data changes. Someone must be responsible for monitoring, retraining decisions, model performance, and business impact.

This creates a complete organizational loop:

Business need → Data → Model → Product experience → Human feedback → Evaluation → Improvement

The goal is not to separate product management, domain expertise, and ML engineering into independent stages. Their value increases when they work together throughout the lifecycle.

This is particularly important for feedback-driven systems. "The Challenge of Feedback Loops in Production Machine Learning" explores how model decisions can influence future data and why production systems need monitoring, human oversight, experimentation, and controlled adaptation.

Key Takeaway

Successful ML products reflect real-world expertise throughout their lifecycle. Human-in-the-loop workflows provide judgment and high-quality feedback, governance helps control risk and accountability, and product and domain teams help identify changing business conditions that models alone may not recognize. The strongest organizations create a continuous partnership between product managers, domain experts, data scientists, and engineers, ensuring that machine learning remains aligned with real user needs, operational realities, and changing business objectives.

Conclusion

Machine learning projects are often presented as technical exercises involving data, algorithms, and model performance. In practice, the hardest part is frequently deciding what the model should actually solve, how its predictions will be used, and whether those predictions reflect the realities of the business or domain.

This is why successful ML projects need product managers and domain experts alongside data scientists and ML engineers.

Product managers help translate broad business objectives into clearly defined decisions, user needs, product requirements, and measurable outcomes. They ensure that the team is solving a problem that matters and that model outputs can be incorporated into a real product workflow.

Domain experts contribute a different but equally important form of knowledge. They understand how the underlying process actually works, which outcomes represent meaningful ground truth, which edge cases matter, and which patterns may be misleading even when they appear statistically significant.

Their involvement can prevent major problems before model development begins.

A poorly defined target can lead the team toward the wrong optimization objective. A misleading label can cause the model to learn incorrect behavior. A seemingly predictive feature may represent leakage or an operational artifact. A technically strong prediction may be impossible to act upon within the real workflow.

Cross-functional collaboration helps identify these issues early.

The strongest teams therefore treat ML development as a shared lifecycle:

Business problem → Data → Target → Features → Model → Product decision → Outcome → Feedback → Improvement

Product managers, domain experts, data scientists, ML engineers, and software engineers contribute different perspectives at each stage.

Collaboration also becomes essential when the team faces unavoidable tradeoffs. Higher recall may increase false positives. A more complex model may improve accuracy but increase latency and cost. Full automation may improve efficiency but increase risk. A technically optimal solution may be operationally impractical.

These decisions cannot be resolved through model metrics alone. They require understanding users, business priorities, domain consequences, and engineering constraints.

Frequently Asked Questions (FAQs)

1. Why do machine learning projects need product managers?

Product managers help define the business problem, identify user needs, establish priorities, connect ML outputs to product workflows, and determine how success should be measured.

2. Why are domain experts important in machine learning?

Domain experts understand the real-world process represented by the data. They can validate labels, identify meaningful patterns, recognize edge cases, and detect assumptions that technical teams may otherwise miss.

3. What does a product manager do on an ML project?

A product manager typically defines the problem and desired outcome, prioritizes requirements, coordinates stakeholders, establishes success metrics, and ensures that the ML capability integrates effectively into the product.

4. What does a domain expert contribute to ML development?

A domain expert contributes knowledge about business processes, terminology, ground truth, edge cases, operational constraints, and the real-world meaning of features and outcomes.

5. Why can technically accurate ML projects still fail?

A model can be technically accurate but still fail because it solves the wrong problem, uses misleading targets, produces predictions that cannot be acted upon, does not fit the workflow, or creates unacceptable cost or risk.

6. How do product managers help define ML success metrics?

They connect technical model metrics to business outcomes such as revenue, retention, conversion, productivity, customer satisfaction, or risk reduction.

7. How do domain experts improve training data and labels?

They can validate whether labels accurately represent the real-world outcome, identify ambiguous cases, correct inconsistencies, and explain exceptions that may not be apparent from the raw data.

8. Why are domain experts important for feature engineering?

They help determine whether a feature has meaningful real-world relevance, whether it represents an operational artifact, whether it could cause leakage, and whether it reflects behavior that should actually influence the prediction.

9. How can product managers and ML engineers work effectively together?

They should align early on the business objective, technical constraints, success metrics, decision workflow, and tradeoffs, while maintaining regular communication throughout development and deployment.

10. Who should own an ML product?

Ownership should be shared across the relevant functions, with clearly defined responsibilities for the product outcome, model, data, infrastructure, monitoring, and operational workflow.

11. How do cross-functional teams handle ML tradeoffs?

Teams should evaluate tradeoffs using both technical and business criteria, including accuracy, precision, recall, latency, cost, operational capacity, user experience, risk, and expected business impact.

12. Why are human-in-the-loop workflows important?

Human review can provide judgment for ambiguous or high-impact cases, improve label quality, identify model failures, and provide feedback that can support future model improvements.

13. How do domain experts help identify ML failure modes?

They can identify rare events, operational exceptions, unrealistic predictions, and situations where statistically plausible model behavior does not make sense in the real-world context.

14. When should product managers and domain experts become involved in an ML project?

They should be involved from the beginning, during problem definition and target selection—and remain engaged through data preparation, model evaluation, product integration, deployment, monitoring, and improvement.

15. What does a strong cross-functional machine learning team look like?

A strong team combines product management, domain expertise, data science, ML engineering, and software engineering, with shared accountability for moving from the business problem to reliable data, useful predictions, actionable decisions, measurable outcomes, and continuous improvement.