Delphi Method

The Delphi method, also known as the Delphi technique, is a structured research method used to collect and progressively refine the judgements of a panel of experts through multiple rounds of questionnaires and controlled feedback. Unlike a conventional survey or a single round of expert interviews, participants reconsider their judgements after receiving summarized feedback from the group while generally remaining anonymous to one another.

Delphi research is particularly useful when a problem requires expert judgement but reliable empirical evidence is limited, fragmented or still emerging. It has been applied to forecasting, priority setting, framework development, identification of future challenges, policy development and other research problems where informed judgement needs to be systematically organized.

There is no universally correct Delphi recipe. Decisions about who qualifies as an expert, how large the panel should be, how many rounds should be conducted and what constitutes consensus need to form part of a coherent research design rather than being copied mechanically from previous studies.

On this page:

• Delphi Method Explained Simply
• What Is the Delphi Method?
• Key Characteristics of the Delphi Method
• How the Delphi Method Works
• Classical vs Modified Delphi
• Selecting Experts for a Delphi Panel
• How Many Experts Are Needed?
• How Many Delphi Rounds Are Needed?
• Consensus in the Delphi Method
• When Should a Delphi Study Stop?
• Dudovskiy Delphi Design Decision Framework
• Delphi Method vs Expert Interviews
• Delphi Method vs Focus Groups
• Analysing Delphi Data
• Application of the Delphi Method: an Example
• Advantages and Limitations of the Delphi Method
• Common Mistakes When Using the Delphi Method
• Delphi Method in Business Research
• Delphi Method in the Age of AI and Digital Research
• When to Use the Delphi Method
• Dissertation Example
• Exam Tip

Delphi Method Explained Simply

Imagine that a researcher wants to identify the most important skills that supply-chain managers will need as artificial intelligence becomes more widely adopted. Existing academic evidence is limited because technologies and managerial practices are changing rapidly. However, experienced supply-chain executives, technology specialists and academics already possess relevant knowledge.

The researcher assembles a panel of suitably qualified experts. In the first round, participants independently identify important future competencies. Their responses are analysed and converted into a structured list, which is returned to the panel in the second round for rating and comment. Participants receive summarized feedback about the group’s responses and are subsequently given an opportunity to reconsider their judgements. Further rounds continue according to the study’s predefined design and stopping logic.

The important point is that Delphi is not simply a survey of experts. Its distinctive value comes from structured iteration, controlled feedback and the opportunity for experts to reconsider their judgements without the social pressures associated with face-to-face group discussion.

Dudovskiy Research Assistant

Not sure if Delphi method is the correct choice for your dissertation?

Enter your topic to receive a free, tailored methodology preview.

Dissertation topic

Generate Free Methodology Preview → Free · No account required

Learn more about Dudovskiy Research Assistant

What Is the Delphi Method?

The Delphi method originated in work conducted by the RAND Corporation in the 1950s and was initially associated with technological and strategic forecasting. Dalkey and Helmer (1963) described an experimental application of the method to expert judgement, and Delphi subsequently spread into fields including business, healthcare, education, public policy and technology forecasting.

Although Delphi designs vary considerably, the method generally involves a panel of people with relevant expertise responding individually to a series of questionnaires. Responses from one round are summarized and fed back to participants, who can maintain or revise their judgements in subsequent rounds. The process therefore differs from collecting independent expert opinions only once.

A widely cited formulation by Linstone and Turoff (1975) conceptualizes Delphi as a structured communication process designed to enable a group to deal effectively with a complex problem. This emphasis on structured communication is important because consensus is not the only possible reason for using Delphi. Depending on the research purpose, researchers may also be interested in identifying priorities, exploring uncertainty, clarifying disagreement or examining the stability of expert judgements.

Key Characteristics of the Delphi Method

Several characteristics distinguish Delphi from other forms of expert consultation. The first is iteration: participants contribute across more than one round rather than providing a single response. The second is controlled feedback, whereby information derived from previous rounds is systematically returned to participants. The third is the opportunity for reconsideration, allowing experts to revise their responses after seeing information about the panel’s collective judgement.

Another important characteristic is anonymity or quasi-anonymity between panel members. Participants generally do not engage in direct face-to-face debate and may not know how particular individuals responded. This reduces the ability of dominant personalities, senior figures or highly vocal participants to control group discussion. It does not necessarily mean that participants are anonymous to the researcher.

Delphi studies also normally involve some form of statistical aggregation or structured synthesis of group responses. Quantitative rounds may report measures such as medians, interquartile ranges, percentages of agreement or distributions of ratings, while qualitative responses may be summarized and categorized before being returned to the panel.

How the Delphi Method Works

A Delphi study normally begins by defining a problem for which expert judgement can make a meaningful contribution. Researchers then establish explicit criteria for expertise and recruit a panel capable of addressing that problem. The composition of this panel should reflect the knowledge required by the research question rather than simply the availability of willing participants.

The first round depends on the form of Delphi being used. In a classical Delphi, participants may receive relatively open questions designed to generate ideas without imposing a predetermined structure. Researchers analyse these responses, combine overlapping ideas and develop items that can be evaluated in subsequent rounds. In a modified Delphi, researchers may begin with a predefined set of items derived from previous research, guidelines, interviews or another evidence source.

Later rounds commonly ask participants to rate, rank or comment on the resulting items. Researchers summarize the responses and provide controlled feedback to the panel. Participants then reconsider their answers in light of this information. The process continues until the study’s stopping criterion has been satisfied or the researchers can otherwise justify terminating the iterative process.

A simplified sequence is:

Research problem → Expert panel → Round 1 → Analysis → Controlled feedback → Round 2 → Reassessment → Further round(s) if justified → Final Delphi outcome

Classical vs Modified Delphi

The distinction between classical Delphi and modified Delphi primarily concerns how the process begins rather than whether later rounds involve expert judgement and feedback.

A classical Delphi typically starts with an open exploratory round. Participants generate issues, predictions, criteria or potential solutions themselves, and the researcher develops subsequent questionnaires from these responses. This approach is useful when the objective is to explore a poorly structured problem without imposing an extensive predefined list.

A modified Delphi begins with existing material. Initial items might come from a systematic or conventional literature review, previous interviews, an established framework, policy documents or another evidence source. Experts then evaluate and refine these items. A modified approach can be more efficient when substantial knowledge already exists, although researchers should consider whether the predefined material unnecessarily constrains the range of ideas the panel can introduce.

The appropriate choice therefore depends on the research problem. A study attempting to discover previously unidentified future risks may benefit from an open first round, whereas a study seeking expert agreement on an existing set of proposed competency standards may have a stronger justification for beginning with predefined items.

Selecting Experts for a Delphi Panel

The quality of a Delphi study depends substantially on whether its participants possess expertise relevant to the problem being investigated. Yet expert is not a self-explanatory sampling category. Researchers need to state what expertise means in the context of their particular study and how potential participants were assessed against those criteria.

Relevant indicators might include professional experience, academic specialization, responsibility for particular organizational decisions, publications, specialist qualifications or direct involvement with the phenomenon under investigation. The appropriate combination depends on the research question. A Delphi study on the future regulation of autonomous vehicles, for example, might deliberately require perspectives from transport regulators, automotive engineers, legal specialists and industry practitioners rather than treating all expertise as interchangeable.

Panel composition also matters because expert judgement can be shaped by professional background and institutional position. A technically impressive panel consisting entirely of one stakeholder group may produce a narrower perspective than the research problem requires. Researchers should therefore justify not only whether individual participants qualify as experts, but also whether the panel collectively contains the range of expertise necessary for the study.

How Many Experts Are Needed for a Delphi Study?

There is no universally accepted minimum or ideal number of Delphi experts. Appropriate panel size depends on factors such as the specificity of the research problem, heterogeneity of required expertise, number of stakeholder groups, accessibility of genuinely qualified participants and expected attrition across rounds.

This makes statements such as “Delphi studies require 15–20 experts” too rigid unless they are explicitly tied to a particular methodological source and research context. Recent methodological research continues to show substantial variation in Delphi panel sizes and limited standardization of sample-size justification.

A defensible dissertation should therefore explain the logic of panel construction rather than presenting a conventional-looking number as sufficient justification. If several distinct stakeholder groups are methodologically necessary, for example, the panel should contain enough relevant expertise to represent those perspectives meaningfully. Conversely, recruiting a very large number of weakly qualified participants does not necessarily strengthen a study whose validity depends on specialist knowledge.

How Many Delphi Rounds Are Needed?

There is similarly no universal number of Delphi rounds. Two or three rounds are common in many applications, but frequency in previous research does not transform these numbers into methodological requirements.

The number of rounds should reflect what the study is trying to accomplish. An exploratory first round may be required to generate ideas, followed by a rating round and a further reassessment after feedback. A modified Delphi beginning with established items may require a different sequence because the item-generation stage has already been completed.

Additional rounds also have costs. Repeated questionnaires increase participant burden and can lead to attrition, while later rounds may produce progressively smaller changes in responses. Researchers should therefore avoid assuming that more rounds automatically make a Delphi study more rigorous. The relevant question is whether another round is likely to contribute methodologically useful information.

Consensus in the Delphi Method

Consensus is commonly associated with Delphi research, but there is no universally accepted statistical definition of Delphi consensus. Diamond et al. (2014), in a systematic review of consensus definitions and stopping criteria, found substantial variation in how Delphi studies defined consensus. Percentage agreement was frequently used, but thresholds differed considerably between studies.

Consensus can be operationalized using percentage agreement, medians and dispersion measures, interquartile ranges, proportions of ratings falling within specified categories, or other predefined rules appropriate to the measurement scale and research objective. The choice should be made deliberately and, where possible, specified before the results are known.

Researchers should also distinguish consensus from unanimity. A Delphi panel can reach a predefined level of agreement while some experts continue to disagree. Those minority positions may remain substantively important, particularly when the research concerns uncertain futures, policy choices or contested professional judgements.

For this reason, reporting only that “consensus was achieved” can conceal valuable information. Stronger reporting explains how consensus was defined, which items satisfied the criterion, how responses changed across rounds and where meaningful disagreement remained.

When Should a Delphi Study Stop?

A predetermined number of rounds is administratively convenient, but it should not automatically substitute for a methodological stopping rationale. A Delphi study might reasonably stop when a predefined consensus criterion has been met, when responses have become sufficiently stable, when additional rounds produce little meaningful change, or when another explicitly justified methodological criterion applies.

Consensus and stability are related but different. Consensus concerns the extent of agreement among panel members, whereas stability concerns the extent to which responses stop changing between rounds. A panel can remain divided but stable, meaning that further rounds may be unlikely to resolve the disagreement. Conversely, apparent agreement at one point does not necessarily show that judgements have stabilized.

The stopping rule should therefore be considered during study design rather than invented after researchers see the results. If consensus is the intended outcome, researchers should specify what will count as consensus and how failure to reach it will be handled. Failure to achieve consensus is not necessarily research failure; persistent disagreement can itself be an important finding.

Dudovskiy Delphi Design Decision Framework

The Dudovskiy Delphi Design Decision Framework is a practical decision aid for constructing a coherent Delphi study. It does not replace established Delphi principles such as iteration, controlled feedback, expert judgement or anonymity. Instead, it organizes the major design decisions researchers need to justify rather than encouraging them to copy apparently standard numbers from previous studies.

The framework begins with the research problem and asks whether structured expert judgement is genuinely appropriate. If it is, the researcher defines the panel logic by determining what constitutes relevant expertise and what perspectives the panel needs to contain. The next decision concerns the starting point: an open exploratory first round may support a classical Delphi, whereas predefined items derived from existing evidence may support a modified design.

Researchers then specify the iterative feedback logic, including what information will be returned to participants and how they will be able to reconsider their judgements. The outcome rule establishes how consensus, disagreement, stability or another relevant outcome will be operationalized. Finally, the stopping rule determines when further rounds cease to provide sufficient methodological value.

The resulting sequence is:

Research Problem → Need for Expert Judgement → Panel Logic → Starting Point → Iterative Feedback → Outcome Rule → Stopping Rule → Defensible Delphi Design

The central principle is:

A rigorous Delphi design is justified by the logic connecting its research problem, experts, rounds, feedback and decision rules—not by reproducing a conventional panel size, number of rounds or consensus percentage.

Delphi Method vs Expert Interviews

Delphi and expert interviews both draw on specialist knowledge, but they structure that knowledge differently. An expert interview normally provides an in-depth account from an individual participant at a particular point in time. Researchers may subsequently compare interviews, but participants do not usually receive systematic group feedback and reconsider their answers through successive rounds.

Delphi deliberately introduces iteration. Experts respond independently, receive controlled information derived from the panel and have an opportunity to revise or maintain their judgements. This makes Delphi particularly suitable when the research objective involves structured collective judgement rather than simply understanding individual expert perspectives.

Neither method is inherently superior. The appropriate choice follows from the research question.

Delphi Method vs Focus Groups

Focus groups generate data through direct interaction among participants. Participants hear each other’s arguments immediately, challenge ideas, develop points collaboratively and may change their views during the discussion. This interaction can produce rich data, but it also creates risks associated with status differences, dominant personalities and conformity pressures.

Delphi separates participants from direct group confrontation. Feedback is mediated by the research process, and participants generally reconsider their judgements privately. This can be particularly valuable when panel members occupy different professional positions or are geographically dispersed.

The distinction again depends on the objective. If the researcher wants to observe and analyse interactive discussion, a focus group may be preferable. If the aim is to obtain iterative expert judgement while limiting interpersonal influence, Delphi may offer a stronger design.

Analysing Delphi Data

Delphi analysis often combines qualitative and quantitative procedures because different rounds can generate different forms of data. An exploratory first round may produce open-text responses that require coding, categorization or thematic synthesis. Researchers may consolidate overlapping ideas and convert them into structured items for subsequent evaluation.

Later rounds frequently generate ordinal ratings or rankings. Depending on the design, researchers may report medians, interquartile ranges, percentages of agreement, distributions of responses and changes between rounds. The statistical procedure should correspond to the response scale and the predefined definition of the study’s outcome rather than being selected merely because another Delphi study used it.

Qualitative comments can remain important during quantitative rounds. Experts may explain why they disagree with a highly rated item, identify ambiguities or propose revisions. Preserving this information can prevent the analysis from reducing expert judgement to a single percentage while ignoring the reasoning behind minority views.

Application of the Delphi Method: an Example

Consider a study investigating the most important organizational capabilities required for successful implementation of circular-economy practices in medium-sized manufacturing companies. Existing literature identifies numerous possible capabilities, but evidence is fragmented and the relative importance of those capabilities remains uncertain.

The researcher could use a modified Delphi because existing scholarship provides a reasonable starting set of items. A purposive panel might include manufacturing executives with direct circular-economy implementation experience, sustainability specialists and academics whose research focuses on circular production systems. Explicit eligibility criteria would establish what level and type of experience qualifies participants for the panel.

In Round 1, experts could evaluate the initial capabilities, propose missing items and comment on unclear definitions. After refinement, Round 2 could ask the panel to rate each capability’s importance using a specified scale. Participants would then receive controlled feedback showing the distribution or summary of panel ratings together with appropriately synthesized comments. A subsequent round could allow them to reconsider their ratings.

Before data collection, the researcher would specify how agreement or stability will be assessed and what conditions justify terminating the process. The resulting methodology would therefore be stronger than simply stating that “20 experts will participate in three Delphi rounds.” Each component of the design would have a reason connected to the research objective.

Advantages and Limitations of the Delphi Method

A major advantage of Delphi is its ability to organize specialist knowledge when conventional empirical evidence is insufficient to resolve a research problem. Participants can contribute from different organizations or locations without attending the same meeting, while controlled feedback allows them to learn from the collective response without requiring direct confrontation. Anonymity between panel members can also reduce some status and dominance effects that occur in face-to-face groups.

The method is particularly useful for complex and emerging problems because it can accommodate uncertainty rather than pretending that definitive evidence already exists. Iteration gives participants time to reflect, and the final results can show not only areas of agreement but also persistent uncertainty or disagreement.

These strengths create important limitations. Delphi findings depend heavily on who is recognized and recruited as an expert, meaning that weak panel construction can undermine the entire study. Successive rounds require considerable time from both researchers and participants, increasing the risk of attrition. Questionnaire design, feedback presentation and decisions about combining or removing items can also introduce researcher influence into what may appear to be a purely expert-driven process.

Consensus itself can become problematic when treated as proof that a proposition is objectively correct. Experts can collectively be mistaken, and convergence may partly reflect the feedback process. A well-designed Delphi study therefore provides structured expert judgement, not certainty or empirical proof.

Common Mistakes When Using the Delphi Method

One of the most common errors is labelling any consultation with experts as Delphi. A one-round questionnaire administered to specialists may constitute an expert survey, but it lacks the iterative feedback and reconsideration normally central to Delphi research.

Another mistake is using methodological numbers without methodological reasoning. Statements such as “20 experts were selected because Delphi requires 15–20 participants,” “three rounds were conducted because Delphi normally uses three rounds,” or “75% was chosen because that is the standard consensus level” present conventions as universal rules when the methodological literature demonstrates considerable variation.

Weak definitions of expertise create a further problem. Recruiting participants because they have convenient job titles does not establish that they possess knowledge relevant to the research question. Researchers should make eligibility criteria explicit and explain why the resulting panel collectively provides the expertise required.

Finally, researchers sometimes treat consensus as the only valuable outcome. Persistent disagreement can reveal competing professional perspectives, unresolved uncertainty or genuine differences between stakeholder groups. Forcing additional rounds simply to manufacture convergence can obscure rather than improve the findings.

Delphi Method in Business Research

The Delphi method is particularly relevant to business and management research when decisions concern emerging technologies, future capabilities, strategic risks or other phenomena for which historical data provide an incomplete guide. Potential applications include forecasting industry developments, identifying future workforce competencies, prioritizing sustainability challenges, evaluating technology-adoption barriers and developing criteria for strategic decision-making.

Business research also illustrates why panel composition deserves careful attention. A study of artificial-intelligence governance might produce very different judgements if its panel consists entirely of technology executives rather than including legal, risk-management, ethics and operational expertise. Heterogeneity can therefore be methodologically valuable when the research problem itself spans multiple forms of specialist knowledge.

Researchers should nevertheless align the claims with the method. A Delphi panel can establish what a specified group of experts judges to be important or likely; it does not automatically establish that their forecast will occur or that their preferred strategy will produce the best organizational outcome.

Delphi Method in the Age of AI and Digital Research

Digital platforms have already made geographically dispersed Delphi panels easier to administer, while AI creates new possibilities for managing large volumes of open-text feedback. AI systems can assist with organizing Round 1 responses, identifying semantically similar suggestions, flagging duplicate items and preparing preliminary summaries for researcher review. These applications could substantially reduce the administrative burden of iterative studies.

The central methodological risk is that controlled feedback is part of the research intervention itself. If an AI system summarizes expert comments inaccurately, removes minority reasoning or combines substantively different ideas because they appear linguistically similar, it can change what panel members encounter in the next round. That can influence subsequent judgements and therefore affect the outcome of the Delphi process.

Researchers using AI should consequently preserve traceability between original expert responses, researcher decisions and the feedback returned to participants. AI-generated clustering or summaries should be checked rather than accepted automatically, particularly where disagreement is methodologically meaningful.

Generative AI also complicates the meaning of expertise. The availability of sophisticated AI-generated answers does not remove the need to justify why particular human participants possess relevant experiential, professional or scholarly knowledge. A Delphi panel should not become a mechanism for collecting generic information that could have been obtained through a literature search or conventional research synthesis. Its value lies in structured judgement from appropriately selected experts.

When to Use the Delphi Method

The Delphi method may be appropriate when:

• the research problem genuinely requires informed expert judgement;
• empirical evidence is incomplete, uncertain or rapidly evolving;
• the objective involves forecasting, prioritization, framework development or systematic evaluation of possible future developments;
• geographically dispersed experts need to contribute to a collective process;
• direct group interaction could create undesirable dominance or status effects;
• participants need opportunities to reconsider their judgements after receiving structured feedback;
• agreement, stability or persistent disagreement among experts is itself analytically useful; or
• the researcher can justify a multi-round process and retain sufficient panel participation across rounds.

Delphi is generally less appropriate when reliable empirical data can answer the research question directly, when the objective is simply to explore individual experiences in depth, or when there is no meaningful reason for participants to reconsider their responses after seeing group feedback.

Dissertation Example

Consider a dissertation titled “Expert Consensus on the Critical Governance Requirements for Generative AI Adoption in European Financial Services.” The methodology chapter could justify the Delphi method because generative AI governance is developing rapidly, relevant empirical evidence remains incomplete, and the research objective is to systematically evaluate informed judgements from specialists working across several relevant domains.

The researcher might purposively recruit experts from financial-services technology, risk management, compliance, AI governance and academic research using predefined experience criteria. A modified Delphi could begin with governance requirements identified from existing literature and regulatory documents. In the first study round, panel members could evaluate these requirements, suggest additional items and identify ambiguous formulations. Subsequent rounds could involve structured ratings, controlled statistical and qualitative feedback, and opportunities to reconsider previous judgements.

The methodology chapter would define the approach to consensus or stability before analysing the final results and explain the stopping logic rather than assuming that exactly three rounds are inherently required. It would also report attrition between rounds and describe how qualitative comments were incorporated into item revisions and participant feedback. The resulting findings could legitimately identify areas of convergence and disagreement among the selected experts, while avoiding the stronger claim that expert consensus proves which governance requirements will objectively be most effective.

Exam Tip

If asked to explain the Delphi method in an examination or dissertation defence, do not describe it simply as “asking experts for their opinions.” A strong answer should identify expert selection, multiple rounds, controlled feedback, reconsideration of responses and systematic synthesis of panel judgements as defining characteristics.

Be particularly prepared to justify the numbers in your methodology. If you propose 18 experts, three rounds and an 80% consensus threshold, an examiner can reasonably ask:

Why 18? Why three? Why 80%?

The strongest answer is not that these numbers are common in Delphi research. It is that each decision has been justified in relation to your research purpose, panel composition, measurement approach and stopping logic.

Designing a Delphi study for your dissertation?
Dudovskiy Research Assistant can evaluate whether the Delphi method fits your research question and help you develop a defensible panel strategy, round structure, feedback process, consensus criterion and stopping rule.

John Dudovskiy

References

Dalkey, N. & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467.

Diamond, I.R., Grant, R.C., Feldman, B.M., Pencharz, P.B., Ling, S.C., Moore, A.M. & Wales, P.W. (2014). Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of Clinical Epidemiology, 67(4), 401–409.

Hasson, F., Keeney, S. & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008–1015.

Hsu, C.-C. & Sandford, B.A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12(10).

Linstone, H.A. & Turoff, M. (Eds.) (1975). The Delphi Method: Techniques and Applications. Addison-Wesley.

Rowe, G. & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15(4), 353–375.

von der Gracht, H.A. (2012). Consensus measurement in Delphi studies: Review and implications for future quality assurance. Technological Forecasting and Social Change, 79(8), 1525–1536.