by Amie Smirthwaite
Clinical evidence strategy is sometimes treated as something to address relatively late in medical device development, once the intended purpose has been defined and the design is largely complete. The difficulty with this approach is that manufacturers may only discover at that stage that the evidence available, or the evidence they are planning to generate, is not sufficient to support their intended claims, clinical benefits or performance expectations. Addressing this can mean revisiting the intended purpose, changing claims, undertaking additional studies or, in some cases, making changes to the device itself. All of this takes time and can delay market access.
Many of these difficulties can be traced back to decisions made much earlier in development. The intended purpose may not have been defined clearly enough, the expected clinical benefits may not have been translated into measurable outcomes, or the chosen endpoints may not adequately support the claims being made. Different functions within an organisation may also be working towards slightly different objectives, without necessarily recognising where their assumptions diverge.
Thinking about clinical evidence requirements earlier in development can help avoid some of these problems. It can also do more than support regulatory compliance. Understanding what needs to be demonstrated, and how, can help inform design decisions, refine the intended purpose and identify where a device is likely to offer the greatest clinical benefit. Clinical evidence strategy should therefore be part of product development, rather than something addressed once the product is essentially finished.
There is also a broader question worth considering. Manufacturers developing devices with similar intended purposes often need to address many of the same fundamental clinical questions. How much of this work genuinely needs to be repeated by each manufacturer, and are there opportunities to make better use of the knowledge that already exists across the industry?
Start with the intended purpose – The story behind the evidence
One area that is often underestimated is the amount of thought that needs to go into defining the intended purpose – beginning in the earliest stages of development, with the identification of the clinical need and design concept.
Intended purpose should address not simply the high-level description of what the device is used for, but the detail underneath it: Which patients? Which indications and clinical circumstances? Are there specific stages or severities of disease, or anatomical locations, for which the device is not suitable? What types of users and use environments are intended? How is the device intended to achieve its effect, and what clinical benefit is it intended to provide? Compared with what, over what timeframe, and using which outcomes? Are there specific sub-populations, limitations, contraindications or use conditions that could affect safety or performance?
The more specificity you can bring to those questions, even if some of it is captured only for internal planning purposes, the clearer the clinical evidence strategy becomes. Precision around the intended purpose (acknowledging that it will almost certainly evolve and be further refined as the device design matures) helps guide the development process by defining what needs to be demonstrated, which evidence is relevant, which endpoints and comparators make sense, and where further evidence may be needed.
A broadly worded intended purpose may seem attractive in terms of preserving commercial flexibility, but it can also create a much larger evidence burden. Every population, clinical benefit or claim implied by that breadth needs to be supported, even ones that are not explicitly stated. If the available evidence does not cover that full breadth, the consequence may be challenge during regulatory or conformity assessment, delay to market access, or pressure to narrow the indication or introduce additional limitations.
More fundamentally, you cannot evaluate whether the evidence is sufficient until you are clear about what the evidence is intended to support. In practice, some apparent evidence gaps are really definition problems: the intended purpose, clinical benefit or associated claim was never specified precisely enough in the first place. What’s more, regulatory trust is built when organisations show their thinking, not when they hide it.
Perhaps that leads to another important question that influences success: Are we trying to prove compliance, or are we trying to demonstrate understanding?
Good clinical evidence strategy depends on cross-functional input
Clinical evidence has implications well beyond the people directly responsible for clinical evaluation, so decisions about clinical strategy and evidence needs should not be made in isolation. Clinical teams, risk management specialists, regulatory professionals, engineering groups, medical affairs and commercial teams all hold pieces of the puzzle. Problems emerge when those pieces are never assembled.
Risk management is a good example. Where risk analysis is driven predominantly from an engineering perspective, clinically important harms or regulatory considerations may be missed. Clinical and Regulatory Affairs input can help ensure that harms are appropriately identified, clinical risks are understood, and the evidence available is sufficient to support the conclusions being drawn. The flow also needs to work in the other direction: new clinical or regulatory information should feed back into risk management as the evidence base develops.
Claims and commercial decisions can have equally important consequences. A relatively small change in wording may broaden an intended purpose or imply a clinical benefit that creates additional evidence requirements. Market Access input may identify further needs around reimbursement, procurement or adoption that go beyond those required for regulatory approval. If those questions are recognised early enough, it may be possible to address several of them through the same evidence-generation activities rather than discovering later that another study or analysis is needed.
The underlying point is that clinical evidence strategy both influences and is influenced by decisions made throughout product development. The organisational structure will differ from company to company, but the important thing is that those decisions are not made without considering their clinical evidence consequences.
What if one of the biggest competitive advantages available to manufacturers today is not faster development cycles, but simply better clinical and regulatory collaboration between functions?
What does “sufficient evidence” actually mean?
One of the recurring difficulties for manufacturers is uncertainty around what evidence is ultimately sufficient for their device.
It’s not simply that the process of defining and justifying sufficiency can be complex. There is also a significant element of judgement and subjectivity. In most cases, there is no neatly defined set of clinical endpoints, acceptance criteria or evidence thresholds that have been agreed across the industry (or clinical expert communities) and can simply be applied. Different experts may reach different conclusions about what constitutes a clinically meaningful outcome or an acceptable level of performance, even when considering the same underlying evidence.
In the EU particularly, that uncertainty is compounded by the fact that a Notified Body cannot act as a consultant to the manufacturer. The MDR does allow exchanges of technical information and regulatory guidance, and structured dialogues have been encouraged to improve the efficiency and predictability of conformity assessment. But those dialogues do not remove the underlying prohibition against the Notified Body telling a manufacturer exactly what evidence they would like to see.
That may create an awkward strategic choice. A manufacturer can generate more evidence than may ultimately be necessary – increasing time and cost – or take a more focused approach and risk discovering later that important questions remain unanswered.
The challenge is therefore not simply to generate “enough” evidence, but to also make a well-reasoned and defensible judgement about what evidence is needed and why. This raises another question:
Where manufacturers are creating these rationales for “sufficiency” for very similar devices, how much of that groundwork should – or really needs to – be recreated from first principles?
Can we build on what the industry already knows?
This is an issue that I think deserves more attention: the amount of clinical evidence work that is repeatedly recreated across the industry.
Manufacturers developing devices with very similar intended purposes may need to answer many of the same fundamental questions, for example:
- What constitutes the relevant state of the art?
- Which clinical endpoints are meaningful?
- What comparator should be used, if any?
- What level of performance is clinically acceptable?
- Which outcomes should form the basis of acceptance criteria?
The answers to those questions can influence study design, sample size, inclusion criteria, follow-up duration, and the way clinical benefit is ultimately demonstrated.
Where standards, common specifications, device-specific guidance or recognised professional consensus provide those answers, manufacturers have a useful starting point. But frequently these kind of specific guidances do not exist. In such cases, each manufacturer may need to work through much of the same groundwork independently.
That creates obvious duplication of effort, but there is another consequence that may be more important: inconsistency. Two manufacturers developing products with essentially the same intended purpose may reach different conclusions about the relevant state of the art, appropriate endpoints, acceptance criteria or even the level of clinical evidence needed.
The absence of a common framework can also make conformity assessment less predictable, because different manufacturers – and potentially different reviewers – may approach very similar clinical questions in different ways.
Some variation will always be justified by differences between devices. But where the underlying clinical question is essentially the same, it is worth asking whether every manufacturer should have to reconstruct the answer from first principles.
How much innovation capacity is being consumed by addressing the same problems over and over again?
There are genuine constraints around intellectual property, commercial sensitivity and ownership of data. But not every aspect of clinical evidence strategy is proprietary. There could be considerable value in identifying those elements that are common across similar device types and considering whether they could be better standardised, structured or shared. That could reduce unnecessary duplication while also improving consistency in how clinical evidence requirements are defined.
The harder question then is: Who should do that work? Manufacturers, regulators, clinical societies, academics and standards bodies all hold part of the answer, and any useful framework would probably need input from several of them.
Evidence strategies have to cope with change
One of the more difficult aspects of lifecycle clinical evidence planning is deciding when change is significant enough to affect the validity of the existing evidence base.
There is particular interest at the moment in how best to manage this for software and AI-enabled devices, given both the rate at which the technology is developing and the potential for the product itself to evolve through updates to algorithms, training data, decision rules, user interfaces or other functionality. Even where the version placed on the market is locked, subsequent updates may create a new version whose clinical performance needs to be considered in relation to the evidence generated for the previous one.
That raises some very practical questions: Does the existing evidence still apply to the updated version? Has previously demonstrated performance been maintained? Could an update affect particular patient subgroups or use conditions differently? And at what point does the scale or nature of the change mean that additional clinical evidence is needed?
Of course this is not just an issue for AI-enabled devices. Clinical evidence can become outdated even when the device itself remains unchanged, because the clinical environment around it may continue to evolve. New therapeutic options or competing technologies may become available, clinical practice may change, professional guidance may be updated, the characteristics of the intended patient population may change, and endpoints or performance expectations that were accepted when the device was first evaluated may no longer represent current clinical practice.
This doesn’t mean that every change requires a new clinical investigation. But what is needed is a proportionate process for deciding when existing evidence remains applicable, when additional verification or clinical performance testing is appropriate, and when the change is significant enough to require new clinical evidence.
An important point to acknowledge is that the post-market evidence generation cannot simply revisit the same questions asked before certification. It should test whether the assumptions underpinning the original clinical evaluation still hold, and whether any changes in the device or its clinical context require the evidence strategy to be revised.
Planning earlier creates more options – and is usually cheaper than fixing things later!
None of this is particularly radical. The challenge is putting the pieces together early enough for them to shape development rather than simply document decisions that have already been made.
A well-designed clinical evidence strategy should create clarity around what needs to be demonstrated, where the main uncertainties lie, and which evidence will be needed by different stakeholders. It should also help teams recognise where decisions made today may create evidence requirements later.
That does not remove uncertainty, but it does give organisations a much better basis for making decisions, allocating resources and adapting when the device, the market or the needs of patients and other stakeholders change.
Good clinical evidence strategy not only reduces the risk of delay and unnecessary rework, but also helps ensure that the evidence generated is useful to regulators, payers, clinicians and patients. That should support better decisions about the product, its place in clinical practice and where further evidence is actually needed.
The future of clinical evaluation may not be about gathering more evidence. It may be about asking better questions earlier, collaborating more effectively and becoming more comfortable navigating uncertainty.
And that is a challenge worth exploring. Contact us today if you’d like to discuss.
