clinical-trials.ai

[1-5] Administrative information

1. Title

Item 1:

Descriptive title identifying the study design, population, interventions, and, if applicable, trial acronym

SPIRIT-AI 1 (i) Elaboration:

Indicate that the intervention involves artificial intelligence/machine learning and specify the type of model.

Indicating in the protocol title and/or abstract that the intervention involves a form of AI is encouraged, as it immediately identifies the intervention as an artificial intelligence/machine learning intervention, and also serves to facilitate indexing and searching of the trial protocol in bibliographic databases, registries, and other online resources. The title should be understandable by a wide audience; therefore a broader umbrella term such as ‘artificial intelligence’ or ‘machine learning’ is encouraged. More precise terms should be used in the abstract, rather than the title, unless broadly recognised as being a form of artificial intelligence/machine learning. Specific terminology relating to the model type and architecture should be detailed in the abstract.

SPIRIT-AI 1 (ii) Elaboration:

State the intended use of the AI intervention.

The intended use of the AI intervention should be made clear in the protocol’s title and/or abstract. This should describe the purpose of the AI intervention and the disease context. Some AI interventions may have multiple intended uses or the intended use may evolve over time. Therefore, documenting this allows readers to understand the intended use of the algorithm at the time of the trial.

2. Trial registration

Item 2a:

Trial identifier and registry name. If not yet registered, name of intended registry

Item 2b:

All items from the World Health Organization Trial Registration Data Set

3. Protocol version

Item 3:

Date and version identifier

4. Funding

Item 4:

Date and version identifier

5. Roles and responsibilities

Item 5a:

Names, affiliations, and roles of protocol contributors

Item 5b:

Name and contact information for the trial sponsor

Item 5c:

Role of study sponsor and funders, if any, in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication, including whether they will have ultimate authority over any of these activities

Item 5d:

Composition, roles, and responsibilities of the coordinating centre, steering committee, endpoint adjudication committee, data management team, and other individuals or groups overseeing the trial, if applicable (see Item 21a for data monitoring committee)

[6-8] Introduction

6. Background and rationale

Item 6a:

Description of research question and justification for undertaking the trial, including summary of relevant studies (published and unpublished) examining benefits and harms for each intervention

SPIRIT-AI 6a (i) Extension:

Explain the intended use of the AI intervention in the context of the clinical pathway, including its purpose and its intended users (e.g. healthcare professionals, patients, public).

In order to understand how the AI intervention will fit into a clinical pathway, a detailed description of its role should be included in the protocol background. AI interventions may be designed to interact with different users including healthcare professionals, patients, and the public, and their roles can be wide-ranging (for example the same AI intervention could theoretically be replacing, augmenting or adjudicating components of clinical decision-making). Clarifying the intended use of the AI intervention and its intended user helps readers understand the purpose for which the AI intervention will be evaluated in the trial.

SPIRIT-AI 6a (ii) Extension:

Describe any pre-existing evidence for the AI intervention.

Authors should describe in the protocol any pre-existing published (with supporting references) or unpublished evidence relating to validation of the AI intervention, or lack thereof. Consideration should be given to whether the evidence was for a similar use, setting and target population as the planned trial. This may include previous development of the AI model, internal and external validations and any modifications made prior to the trial.

Item 6b:

Explanation for choice of comparators

7. Objectives

Item 7:

Specific objectives or hypotheses

8. Trial design

Item 8:

Description of trial design including type of trial (eg, parallel group, crossover, factorial, single group), allocation ratio, and framework (eg, superiority, equivalence, noninferiority, exploratory)

[9-15] Methods: Participants, interventions, and outcomes

9. Study setting

Item 9:

Description of study settings (e.g., community clinic, academic hospital) and list of countries where data will be collected. Reference to where list of study sites can be obtained

SPIRIT-AI 9 Extension:

Describe the onsite and offsite requirements needed to integrate the AI intervention into the trial setting.

There are limitations to the generalisability of AI algorithms, one of which is when they are used outside of their development environment. AI systems are dependent on their operational environment and the protocol should provide details of the hardware and software requirements to allow technical integration of the AI intervention at each study site. For example, it should be stated if the AI intervention requires vendor-specific devices, if there is a need for specialised computing hardware at each site, or if the sites must support cloud integration, particularly if this is vendor-specific. If any changes to the algorithm are required at each study site as part of the implementation procedure (such as fine-tuning the algorithm on local data), then this process should also be clearly described.

10. Eligibility criteria

Item 10:

Inclusion and exclusion criteria for participants. If applicable, eligibility criteria for study centres and individuals who will perform the interventions (eg, surgeons, psychotherapists)

SPIRIT-AI 10 (i) Elaboration:

State the inclusion and exclusion criteria at the level of participants.

The inclusion and exclusion criteria should be defined at the participant level as per usual practice in protocols of non-AI interventional trials. This is distinct from the inclusion and exclusion criteria made at the input data level, which is addressed in item 10 (ii).

SPIRIT-AI 10 (ii) Extension:

State the inclusion and exclusion criteria at the level of the input data.

Input data refer to the data required by the AI intervention to serve its purpose (e.g. for a breast cancer diagnostic system, the input data could be the unprocessed or vendor-specific post-processing mammography scan upon which a diagnosis is being made; for an early warning system, the input data could be physiological measurements or laboratory results from the electronic health record). The trial protocol should pre-specify if there are minimum requirements for the input data (such as image resolution, quality metrics or data format), which would determine pre-randomisation eligibility. It should specify when, how and by whom this will be assessed. For example, if a participant met the eligibility criteria for lying flat for a CT scan as per item 10 (i), but the scan quality was compromised (for any given reason) to such a level that it is no longer fit for use by the AI system, this should be considered as an exclusion criterion at the input data level. Note that where input data are acquired after randomisation (addressed by SPIRIT-20c), any exclusion is considered to be from the analysis, not from enrollment (Figure 1).

Figure 1:

11. Interventions

Item 11a:

Interventions for each group with sufficient detail to allow replication, including how and when they will be administered

SPIRIT-AI 11a (i) Extension:

State which version of the AI algorithm will be used.

Similar to other forms of software as a medical device, AI systems are likely to undergo multiple iterations and updates in their lifespan. The protocol should state which version of the AI system will be used in the clinical trial, and whether this is the same version that has been used in previous studies that have been used to justify the study rationale. If applicable, the protocol should describe what has changed between the relevant versions and the rationale for the changes. Where available, the protocol should include a regulatory marking reference, such as an Unique Device Identifier (UDI) which requires a new identifier for updated versions of the device.

SPIRIT-AI 11a (ii) Extension:

Specify the procedure for acquiring and selecting the input data for the AI intervention.

The measured performance of any AI system may be critically dependent on the nature and quality of the input data. The procedure for how input data will be handled, including data acquisition, selection and pre-processing prior to analysis by the AI system should be provided. Completeness and transparency of this process is integral to feasibility assessment and to future replication of the intervention beyond the clinical trial. It will also help to identify whether input data handling procedures will be standardised across trial sites.

SPIRIT-AI 11a (iii) Extension:

Specify the procedure for assessing and handling poor quality or unavailable input data.

As with 10 (ii), input data refer to the data required by the AI intervention to serve its purpose. As noted in item SPIRIT-AI 10 (ii), the performance of AI systems may be compromised as a result of poor quality or missing input data (for example, excessive movement artefact on an electrocardiogram). The study protocol should specify if and how poor quality or unavailable input data will be identified and handled. The protocol should also specify a minimum standard required for the input data, and the procedure for when the minimum standard is not met (including the impact on, or any changes to, the participant care pathway).

Poor quality or unavailable data can also affect non-AI interventions. For example, sub-optimal quality of a scan could impact a radiologist’s ability to interpret it and make a diagnosis. It is therefore important that this information is reported equally for the control intervention, where relevant. If this minimum quality standard is different from the inclusion criteria for input data used to assess eligibility pre-randomisation, this should be stated.

SPIRIT-AI 11a (iv) Extension:

Specify whether there is human-AI interaction in the handling of the input data, and what level of expertise is required for users.

A description of the human-AI interface and the requirements for successful interaction when handling input data should be described. Examples include clinician-led selection of regions of interest from a histology slide which is then interpreted by an AI diagnostic system, or endoscopist selection of a colonoscopy video clips as input data for an algorithm designed to detect polyps. A description of any planned user training and instructions for how users will handle the input data provides transparency and replicability of trial procedures. Poor clarity on the human-AI interface may lead to a lack of a standard approach and carry ethical implications, particularly in the event of harm. For example, it may become unclear whether an error case occurred due to human deviation from the instructed procedure, or if it was an error made by the AI system.

SPIRIT-AI 11a (v) Extension:

Specify the output of the AI intervention.

The output of the AI intervention should be clearly defined in the protocol. For example, an AI system may output a diagnostic classification or probability, a recommended action, an alarm alerting to an event, an instigated action in a closed-loop system (such as titration of drug infusions), or other. The nature of the AI intervention's output has direct implications on its usability and how it may lead to downstream actions and outcomes.

SPIRIT-AI 11a (vi) Extension:

Explain the procedure for how the AI intervention’s outputs will contribute to decision-making or other elements of clinical practice.

Since health outcomes may also critically depend on how humans interact with the AI intervention, the trial protocol should explain how the outputs of the AI system are used to contribute to decision-making or other elements of clinical practice. This should include adequate description of downstream interventions which can impact outcomes. As with SPIRIT 11a (iv), any elements of human-AI interaction on the outputs should be described in detail. Including the level of expertise required to understand the outputs and any training/instructions provided for this purpose. For example, a skin cancer detection system that produces a percentage likelihood as output should be accompanied by an explanation of how this output should be interpreted and acted upon by the user, specifying both the intended pathways (e.g. skin lesion excision if the diagnosis is positive) and the thresholds for entry to these pathways (e.g. skin lesion excision if the diagnosis is positive and the probability is >80%). The information produced by comparator interventions should be similarly described, alongside an explanation of how such information was used to arrive at clinical decisions for patient management, where relevant.

Item 11b:

Criteria for discontinuing or modifying allocated interventions for a given trial participant (eg, drug dose change in response to harms, participant request, or improving/worsening disease)

Item 11c:

Strategies to improve adherence to intervention protocols, and any procedures for monitoring adherence (eg, drug tablet return, laboratory tests)

Item 11d:

Relevant concomitant care and interventions that are permitted or prohibited during the trial

12. Outcomes

Item 12:

Primary, secondary, and other outcomes, including the specific measurement variable (eg, systolic blood pressure), analysis metric (eg, change from baseline, final value, time to event), method of aggregation (eg, median, proportion), and time point for each outcome. Explanation of the clinical relevance of chosen efficacy and harm outcomes is strongly recommended

13. Participant timeline

Item 13:

Time schedule of enrolment, interventions (including any run-ins and washouts), assessments, and visits for participants. A schematic diagram is highly recommended

14. Sample size

Item 14:

Estimated number of participants needed to achieve study objectives and how it was determined, including clinical and statistical assumptions supporting any sample size calculations

15. Recruitment

Item 15:

Strategies for achieving adequate participant enrolment to reach target sample size

[16-17] Methods: Assignment of interventions (for controlled trials)

16. Allocation

Item 16a: Sequence generation

Method of generating the allocation sequence (eg, computergenerated random numbers), and list of any factors for stratification. To reduce predictability of a random sequence, details of any planned restriction (eg, blocking) should be provided in a separate document that is unavailable to those who enrol participants or assign interventions

Item 16b: Allocation concealment mechanism

Mechanism of implementing the allocation sequence (eg, central telephone; sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the sequence until interventions are assigned

Item 16c: Implementation

Who will generate the allocation sequence, who will enrol participants, and who will assign participants to interventions

17. Blinding (masking)

Item 17a:

Who will be blinded after assignment to interventions (eg, trial participants, care providers, outcome assessors, data analysts), and how

Item 17b:

If blinded, circumstances under which unblinding is permissible, and procedure for revealing a participant’s allocated intervention during the trial

[18-20] Methods: Data collection, management, and analysis

18. Data collection methods

Item 18a:

Plans for assessment and collection of outcome, baseline, and other trial data, including any related processes to promote data quality (eg, duplicate measurements, training of assessors) and a description of study instruments (eg, questionnaires, laboratory tests) along with their reliability and validity, if known. Reference to where data collection forms can be found, if not in the protocol

Item 18b:

Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols

19. Data management

Item 19:

Plans for data entry, coding, security, and storage, including any related processes to promote data quality (eg, double data entry; range checks for data values). Reference to where details of data management procedures can be found, if not in the protocol

20. Statistical methods

Item 20a:

Statistical methods for analysing primary and secondary outcomes. Reference to where other details of the statistical analysis plan can be found, if not in the protocol

Item 20b:

Methods for any additional analyses (eg, subgroup and adjusted analyses)

Item 20c:

Definition of analysis population relating to protocol non-adherence (eg, as randomised analysis), and any statistical methods to handle missing data (eg, multiple imputation)

[21-23] Methods: Monitoring

21. Data monitoring

Item 21a:

Composition of data monitoring committee (DMC); summary of its role and reporting structure; statement of whether it is independent from the sponsor and competing interests; and reference to where further details about its charter can be found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed

Item 21b:

Description of any interim analyses and stopping guidelines, including who will have access to these interim results and make the final decision to terminate the trial

22. Harms

Item 22:

Plans for collecting, assessing, reporting, and managing solicited and spontaneously reported adverse events and other unintended effects of trial interventions or trial conduct

SPIRIT-AI 22 Extension:

Specify any plans to identify and analyse performance errors. If there are no plans for this, explain why not.

Reporting performance errors and failure case analysis is especially important for AI interventions. AI systems can make errors which may be hard to foresee, but which if allowed to be deployed at scale could have catastrophic consequences. Therefore, identifying cases of error and defining risk mitigation strategies is important for informing when the intervention can be safely implemented, and for which populations. The protocol should specify whether there are any plans to analyse performance errors. If there are no plans for this, a justification should be included in the protocol.

23. Auditing

Item 23:

Frequency and procedures for auditing trial conduct, if any, and whether the process will be independent from investigators and the sponsor

[24-31] Ethics and dissemination

24. Research ethics approval

Item 24:

Plans for seeking research ethics committee/institutional review board (REC/IRB) approval

25. Protocol amendments

Item 25:

Plans for communicating important protocol modifications (eg, changes to eligibility criteria, outcomes, analyses) to relevant parties (eg, investigators, REC/IRBs, trial participants, trial registries, journals, regulators)

26. Consent or assent

Item 26a:

Who will obtain informed consent or assent from potential trial participants or authorised surrogates, and how (see Item 32)

Item 26b:

Additional consent provisions for collection and use of participant data and biological specimens in ancillary studies, if applicable

27. Confidentiality

Item 27:

How personal information about potential and enrolled participants will be collected, shared, and maintained in order to protect confidentiality before, during, and after the trial

28. Declaration of interests

Item 28:

Financial and other competing interests for principal investigators for the overall trial and each study site

29. Access to data

Item 29:

Statement of who will have access to the final trial dataset, and disclosure of contractual agreements that limit such access for investigators

SPIRIT-AI 29 Extension:

State whether and how the AI intervention and/or its code can be accessed, including any restrictions to access or re-use.

The protocol should make clear whether and how the AI intervention and/or its code can be accessed or re-used. This should include details regarding the license and any restrictions to access.

30. Ancillary and post-trial care

Item 30:

Provisions, if any, for ancillary and post-trial care, and for compensation to those who suffer harm from trial participation

31. Dissemination policy

Item 31a:

Plans for investigators and sponsor to communicate trial results to participants, healthcare professionals, the public, and other relevant groups (eg, via publication, reporting in results databases, or other data sharing arrangements), including any publication restrictions

Item 31b:

Authorship eligibility guidelines and any intended use of professional writers

Item 31c:

Plans, if any, for granting public access to the full protocol, participant-level dataset, and statistical code

[32-33] Appendices

32. Informed consent materials

Item 32:

Plans, if any, for granting public access to the full protocol, participant level dataset, and statistical code

33. Biological specimens

Item 33:

Plans for collection, laboratory evaluation, and storage of biological specimens for genetic or molecular analysis in the current trial and for future use in ancillary studies, if applicable

Citation

When referring to the SPIRIT-AI guidelines, please cite one of the following articles:

Nature Medicine

The Lancet Digital Health

BMJ