Expanding the Opioids System Model to more comprehensively incorporate fentanyl, stimulants use, polysubstance use and associated outcomes

Expanding the Opioids System Model to more comprehensively incorporate fentanyl, stimulants use, polysubstance use and associated outcomes is sponsored by Food and Drug Administration (FDA). This opportunity supports mission-aligned projects and measurable outcomes.

Expired RFA-FD-21-022: Expanding the opioids system modeling efforts to more comprehensively address fentanyl, stimulants use, polysubstance use and associated outcomes. (U01) Clinical Trials Not Allowed This notice has expired. Check the NIH Guide for active opportunities and notices.

Department of Health and Human Services Part 1. Overview Information Participating Organization(s) U.S. Food and Drug Administration ( FDA ) NOTE: The policies, guidelines, terms, and conditions stated in this announcement may differ from those used by the NIH.

Where this Funding Opportunity Announcement (FOA) provides specific written guidance that may differ from the general guidance provided in the grant application form, please follow the instructions given in this FOA. The FDA does not follow the NIH Page Limitation Guidelines or the NIH Review Criteria.

Applicants are encouraged to consult with FDA Agency Contacts for additional information regarding page limits and the FDA Objective Review Process. of Participating Organizations Center for Drug Evaluation and Research ( CDER ) Funding Opportunity Title Expanding the opioids system modeling efforts to more comprehensively address fentanyl, stimulants use, polysubstance use and associated outcomes.

(U01) Clinical Trials Not Allowed Research Project Cooperative Agreements Funding Opportunity Announcement (FOA) Number Companion Funding Opportunity Additional Information on Eligibility .

Assistance Listing Number(s) Funding Opportunity Purpose The purpose of this funding opportunity is to expand on the FDA's modeling efforts to more comprehensively address fentanyl, stimulants and polysubstance use and their interactive effects on opioids use, misuse, use disorder, addiction treatment, and associated outcomes.

The primary purpose of this project is to capture the dynamics of and interactions between stimulant or polysubstance use and opioid overdose deaths, as an important emerging aspect of the opioids crisis. The research should also seek to quantify how polysubstance use and synthetic opioids may influence the effects of harm reduction approaches and other potential overdose-prevention strategies.

A secondary purpose of the research is to identify and implement feasible approaches for incorporating mental health into part of FDA's modeling Open Date (Earliest Submission Date) Letter of Intent Due Date(s) May 17, 2021, by 11:59 PM Eastern Time.

Applicants are encouraged to apply early to allow adequate time to make any corrections to errors found in the application during the submission Applicants should be aware that on-time submission means that an application is submitted error free (of both Grants. gov and eRA Commons errors) by 11:59 PM Eastern Time on the application due date. applications will not be accepted for this FOA.

AIDS Application Due Date(s) It is critical that applicants follow the Research (R) Instructions (R&R) Application Guide , except where instructed to do otherwise (in this FOA or in a Notice from the NIH Guide for Grants and Contracts ). Conformance to all requirements (both in the Application Guide and the FOA) is required and strictly enforced.

Applicants must read and follow all application instructions in the Application Guide as well as any program-specific instructions noted in Section IV . When the program-specific instructions deviate from those in the Application Guide, follow the program-specific instructions. Applications that do not comply with these instructions may be delayed or not accepted for review.

Part 1. Overview Information Part 2. Full Text of the Announcement I.

Funding Opportunity Description Section II. Award Information Section III. Eligibility Information Section IV.

Application and Submission Section V. Application Review Information Section VI. Award Administration Information Section VII.

Agency Contacts Section VIII. Other Information Full Text of Announcement Section I. Funding Opportunity In 2018, FDA's Center for Drug Evaluation and Research (CDER) began an effort to develop a systems model of the opioid crisis.

The purpose of the model is to help FDA better understand a) the complexity of the underlying mechanisms of the crisis, and b) potential short- and long-term impacts of possible policy actions both desired outcomes and unintended consequences. FDA intends to leverage the model as a practical tool to inform FDA priority-setting and decision-making about potential policies.

Given the model’s integrative perspective, this effort may eventually provide insights into policy-making beyond FDA. FDA's opioids systems model project coincides with complementary research and modeling efforts undertaken by both the National Institute on Drug Abuse (NIDA) and the Centers for Disease Control and Prevention (CDC). An overview of FDA's modeling effort is available at https://healthpolicy.

duke. edu/sites/default/files/2020-11/Introduction%20to%20FDA%27s%20Opioid%20Systems%20Model_White%20Paper. pdf FDA's opioids systems model employs a system dynamic modeling approach [ [1] ], which is particularly well-suited to the opioid crisis for two reasons.

First, the opioid crisis is complex, with multiple interacting feedbacks, time delays, and dynamics that evolve over time and lead to unforeseen or unintended outcomes. A systems model can provide the integrative perspective needed to guide regulatory decision-making under such circumstances. Second, there are large gaps and uncertainties in data relevant to the crisis.

Simulation models allow rapid and extensive sensitivity analyses to help address these uncertainties. The formal architecture of FDA's opioids systems model has taken shape based on a comprehensive review of the literature and numerous interviews with experts in government, academia, and professional practice.

The model focuses on trajectories of opioid use, including, but not limited to: initial exposure to prescription opioids; opioid use, abuse, and misuse; development of opioid use disorder (OUD); use of illicit opioids; opioid overdose, death, and harm reduction; and OUD treatment and relapse. The model identifies key influences on these processes, including situational, behavioral, and motivational factors.

It depicts the U.S. population, allowing FDA and other policy makers to consider the crisis on a national scale. In so doing, it does not account for geographic variability in population or effect FDA's opioids systems model is built upon academically rigorous standards and careful assessment of the best available evidence of the opioids system.

FDA's opioids systems model will be fully transparent and documented, as well as publicly available, for further reproducibility and validation. At its initial stage, the systems model is considered a baseline dynamic model limited in current scope and quantification. The model has been developed to enable continual improvements, expansion, and adaptation as the complexities of the crisis change.

There is a continual need to enhance the model by refining the underlying model structure, refining the quantification of the model parameters, expanding the scope as appropriate, and transforming the model into a practical policy analysis and communication tool. Model development began internally in 2018.

In September 2019, FDA awarded a grant under RFA-FD-19-026 to continue enhancing and improving FDA's systems model mainly focusing on the following areas: 1- Enhance modeling of utilization, effectiveness, and outcomes of treatment for opioid use disorder (such as Medication-Assisted Treatment (MAT) and behavioral therapy); 2- Enhance modeling of opioids prescribing and prescription opioids utilization; 3- Research and enhance the modeling of potential interventions of most interest to FDA.

In August 2020, FDA awarded a second grant under RFA-FD-20-031 to explicitly incorporate a) social outcomes (e.g. quality of life, untreated pain, and mental health and suicide) and b) cost and/or cost effectiveness of policy interventions to reduce or mitigate the negative impacts of the opioids crisis (e.g. FDA regulatory changes, development of new therapies, changing prescribing practices, increase adoption of opioids use disorder treatment approaches, and enhancement of prevention and harm reduction strategies) into FDA's systems model and model analyses.

RFA-FD-20-031 is anticipated to conclude in September 2021. FDA has also awarded a two-year contract (September 2019 - September 2021). Key objectives of that contract include conducting a third-party review and validation of the model (as of October 2020) and supporting development of a policy tool and accompanying policy analysis service that can fit within FDA's scientific and regulatory decision making processes.

The current efforts are expected to result in a systems model and policy analysis tool to explore the intended and unintended consequences of potential opioids policy options. Over the last two years, the collective efforts of the FDA and our collaborators have focused on model development and quantification.

With the completion of an initial version of the model imminent, we are shifting focus to exploratory use of the model within FDA and incorporation of model enhancements. Our intent is to continue to enhance the model as the crisis and our collective understanding of it evolves.

With opioid over dose death on the rise partly due to synthesized opioids and polysubstance use in the more recent years, one of the limitations of the current model is lack of in-depth consideration for complexities of the evolving opioids/stimulants crisis and policies that may address aspects of the crisis in terms of polysubstance use and harm reduction, as well as considerations for mental health comorbidities.

We seek to fill this gap with this research. FDA seeks research that can supplement its opioids systems modeling efforts. The purpose of this funding opportunity is to focus on two important but complex areas that to date have not been incorporated into the model.

The first and highest priority is fentanyl, stimulants and poly-substance use. This includes their interactive effects on opioids use, misuse, use disorder, use disorder treatment, and associated outcomes into the existing model and the corresponding policy analysis tool.

The second is identify and incorporate aspects of mental health comorbidities that are feasible for inclusion into the model and that contributes to areas of opioids use, polysubstance use, opioid use disorder, treatment and/or overdose. It is clear that effectively addressing the opioids crises must simultaneously address these complex aspects of the crisis.

With the proliferation of illicit synthetics, an increasing number of opioid overdose deaths involves people who are primarily stimulant or polysubstance users. With stimulant use on the rise, these populations are an important emerging frontier in the opioid crisis and public health more generally.

Harm-reduction approaches aimed at addressing opioid mortality, such as naloxone distribution targeted at heroin users or attempts to increase the availability of treatment for opioid use disorders, may miss these increasingly important user populations and have diminishing effectiveness as a result.

Quantifying the overlap and interaction between stimulant use and synthetic opioid overdose is crucial to addressing this emerging problem. Work resulting from this funding opportunity should provide more holistic insight into the opioid crisis and enhance decision makers' ability to propose and consider policy options.

Such work will require comprehensive research of underlying dynamics of said effects and to the extent possible integration of new structural components, new data, and new analyses into the model. This research should focus on two areas.

First, expanding the FDA's modeling efforts to more comprehensively address fentanyl, stimulants, and poly-substance use and their interactive effects on opioids use, misuse, addiction, use disorder treatment, and associated outcomes. This translates to studying the growing number of opioid overdose deaths involving people who are primarily stimulant or polysubstance users, as an important emerging aspect of the opioid's crisis.

Additionally, the research will include quantification of the effects of harm reduction approaches (e.g., naloxone distribution to opioid users, increasing treatment availability for OUD) on overdoses involving fentanyl and other polysubstance use. The second area of focus is the inclusion of relevant aspects of mental health comorbidities.

Achieving these research objectives requires bringing together input from a wide range of relevant public and private data sources, as well as subject matter experts. This input process should be timely, well-informed, candid, thoughtful, thorough, and well-documented.

FDA is therefore seeking to establish a cooperative agreement with a qualified awardee with experience in the conduct of the needed research, workshops and other meetings as necessary, and related work. The awardee will conduct research and engage in model building and quantification activities.

Such activities will expand the capability of FDA's model to incorporate effects of fentanyl, stimulants and poly-substance use and their interactive effects on opioids use, misuse, addiction, use disorder treatment, and associated outcomes. Where expert interviews are needed, the awardee will conduct background research prior to expert engagement and use input gathered to develop models and prepare white papers and research reports.

FDA's current dynamic model has been built using system dynamics modeling formalism and has been executed in Vensim software platform developed by Ventana Systems (vensim. com). Any structural changes and sub-models produced as result of this research may be integrated or feed into FDA's model to the extent possible and be compatible with it.

These additions and improvements to FDA's current model will take place with FDA input and discussion. The awardee's research project must align with other research projects undertaken as part of FDA's opioids systems model initiative. (FDA will be responsible for facilitating coordination.)

Inherent in the cooperative agreement award is substantive involvement by the awarding agency. Accordingly, FDA will have substantive involvement in the programmatic activities funded by this cooperative agreement. Substantive involvement includes, but is not limited to, guidance and assistance in the identification of key research objectives and the provision of subject matter expertise.

FDA will also work closely with the research team on any relevant quantification of the model and policy evaluation analysis VIII. Other Information for award authorities and regulations. Cooperative Agreement: A support mechanism used when there will be substantial Federal scientific or programmatic involvement.

Substantial involvement means that, after award, FDA scientific or program staff will assist, guide, coordinate, or participate in project activities. See Section VI. 2 for additional information about the substantial involvement for Application Types Allowed Glossary and the SF424 (R&R) Application Guide provide details on these application types.

Only those application types listed here are allowed for Not Allowed: Only accepting applications that do not help determining whether you are doing a clinical trial? Funds Available and Anticipated Number of Awards The number of awards is contingent upon FDA appropriations and the submission of a sufficient number of meritorious applications. Award(s) will provide one (1) year of support.

FDA /CDER} intends to commit up to $1,250,000 in FY 2021 It is anticipated that ONE (1) award will be made, not to exceed $1,250,000 in total costs (direct plus indirect). Application budgets need to reflect the actual needs of the proposed project. and should not exceed the following in total costs (direct The scope of the proposed project should determine the project period.

The maximum project period is ONE (1) year. Grants Policy Statement will apply to the applications submitted and awards made from this FOA.

Higher Education Institutions Public/State Controlled Institutions of Higher Education Private Institutions of Higher Education The following types of Higher Education Institutions are always encouraged to apply for FDA support as Public or Private Institutions of Higher Education: Hispanic-serving Institutions Historically Black Colleges and Universities (HBCUs) Tribally Controlled Colleges and Universities (TCCUs) Alaska Native and Native Hawaiian Serving Institutions Asian American Native American Pacific Islander Serving Institutions Nonprofits Other Than Institutions of Higher Education Nonprofits with 501(c)(3) IRS Status (Other than Institutions of Nonprofits without 501(c)(3) IRS Status (Other than Institutions For-Profit Organizations (Other than Small Businesses) City or Township Governments Special District Governments Indian/Native American Tribal Governments (Federally Recognized) Indian/Native American Tribal Governments (Other than Federally U.S. Territory or Possession Independent School Districts Public Housing Authorities/Indian Housing Authorities Native American Tribal Organizations (other than Federally recognized tribal governments) Faith-based or Community-based Organizations Non-domestic (non-U.S.) Entities (Foreign Institutions) are Non-domestic (non-U.S.) components of U.S. Organizations are not eligible Foreign components, as defined in the HHS Grants Policy Statement , are not allowed.

Applicant organizations must complete and maintain the following registrations as described in the SF 424 (R&R) Application Guide to be eligible to apply for or receive an award. All registrations must be completed prior to the application being submitted. Registration can take 6 weeks or more, so applicants should begin the registration process as soon as possible.

Failure to complete registrations in advance of a due date is not a valid reason for a Universal Numbering System (DUNS) - All registrations require that applicants be issued a DUNS number. After obtaining a DUNS number, applicants can begin both SAM and eRA Commons registrations. The same DUNS number must be used for all registrations, as well as on the grant application.

System for Award Management (SAM) Applicants must complete and maintain an active registration, which requires renewal at least annually . The renewal process may require as much time as the initial registration. SAM registration includes the assignment of a Commercial and Government Entity (CAGE) Code for domestic organizations which have not already been assigned a CAGE Code.

and Government Entity (NCAGE) Code Foreign organizations must obtain an NCAGE code (in lieu of a CAGE code) in order to register in SAM. must have an active DUNS number to register in eRA Commons. Organizations can register with the eRA Commons as they are working through their SAM or Grants.

gov registration, but all registrations must be in place by time of submission. eRA Commons requires organizations to identify at least one Signing Official (SO) and at least one Program Director/Principal Investigator (PD/PI) account in order to submit an application. must have an active DUNS number and SAM registration in order to complete the Grants.

gov Directors/Principal Investigators (PD(s)/PI(s)) All PD(s)/PI(s) must have an eRA Commons account. PD(s)/PI(s) should work with their organizational officials to either create a new account or to affiliate their existing account with the applicant organization in eRA Commons. If the PD/PI is also the organizational Signing Official, they must have two distinct eRA Commons accounts, one for each role.

Obtaining an eRA Commons account can take up to 2 weeks. Eligible Individuals (Program Director/Principal Any individual(s) with the skills, knowledge, and resources necessary to carry out the proposed research as the Program Director(s)/Principal Investigator(s) (PD(s)/PI(s)) is invited to work with his/her organization to develop an application for support.

Individuals from underrepresented racial and ethnic groups as well as individuals with disabilities are always encouraged to apply for FDA support. For institutions/organizations proposing multiple PDs/PIs, visit the Multiple Program Director/Principal Investigator Policy and submission details in the Senior/Key Person Profile (Expanded) Component of the SF424 (R&R) This FOA does not require cost sharing as defined in the HHS 3.

Additional Information on Eligibility Applicant organizations may submit more than one application, provided that each application is scientifically distinct. The FDA will not accept duplicate or highly overlapping applications under review at the same time.

This means that the FDA will A new (A0) application that is submitted before issuance of the summary statement from the review of an overlapping new (A0) or resubmission A resubmission (A1) application that is submitted before issuance of the summary statement from the review of the previous new (A0) application. Section IV.

Application and Submission Information The application forms package specific to this opportunity must be accessed through ASSIST, Grants. gov Workspace or an institutional system-to-system solution. Links to apply using ASSIST or Grants.

gov Workspace are available in Part 1 of this FOA. See your administrative office for instructions if you plan to use an institutional system-to-system 2. Content and Form of Application Submission It is critical that applicants follow the Research (R) Instructions (R&R) Application Guide , except where instructed in this funding opportunity announcement to do otherwise.

Conformance to the requirements in the Application Guide is required and strictly enforced.

Applications that are out of compliance with these instructions may be delayed or not accepted for Although a letter of intent is not required, is not binding, and does not enter into the review of a subsequent application, the information that it contains allows FDA staff to estimate the potential review workload and By the date listed in Part 1.

Overview Information , prospective applicants are asked to submit a letter of intent that includes the following information: Descriptive title of proposed activity Name(s), email address(es), and telephone number(s) of the Names of other key personnel Participating institution(s) Number and title of this funding opportunity The letter of intent should be sent to: Food and Drug Administration/Center for Drug Evaluation and Research/Office of the Commissioner/Office of Operations/Office of Finance, Budget and Acquisitions/Office of Acquisitions and Grants Services A technical session will be held for prospective applicants in APRIL 2021.

The conference call information will be provided to prospective applicants that submit a letter of intent. The technical session will provide an overview of the submission requirements and allow prospective applicants an opportunity to ask questions regarding the application process. Participation in the technical session is optional, but strongly encouraged.

All page limitations described in the SF424 Application Page Limits must be followed.

with the following exceptions or additional For this specific FOA, the Research Strategy section is Instructions for Application Submission The following section supplements the instructions found in the SF424 (R&R) Application Guide and should be used for preparing an All instructions in the SF424 (R&R) Application Guide SF424(R&R) Project/Performance Site Locations All instructions in the SF424 (R&R) Application Guide SF424(R&R) Other Project Information All instructions in the SF424 (R&R) Application Guide SF424(R&R) Senior/Key Person Profile All instructions in the SF424 (R&R) Application Guide All instructions in the SF424 (R&R) Application Guide must be followed, with the following additional instructions: Applications requesting multiple years of support must complete and submit a separate detailed budget breakdown and narrative justification for each year of financial support requested.

If an applicant is requesting indirect costs as part of their budget, a copy of the most recent Federal indirect cost rate or F&A agreement must be provided as part of the application submission. This agreement should be attached to the RESEARCH & RELATED Other Project Information Component as line #12 'Other Attachments'.

If the applicant organization has never established an indirect cost rate and/or does not have a negotiated Federal indirect cost rate agreement, a de minimis indirect cost rate of 10 percent (10%) of modified total direct costs (MTDC) will be allowed.

MTDC means all direct salaries and wages, applicable fringe benefits, materials and supplies, services, travel, and subaward and subcontracts up to the first $25,000 of each subaward or subcontract. MTDC excludes equipment, capital expenditures, charges for patient care, rental costs, tuition remission, scholarships and fellowships, participant support costs and the portion of each subaward and subcontract in excess of $25,000.

Indirect/F&A costs under grants to foreign and international organizations will be funded at a fixed rate of 8 percent of modified total direct costs (MTDC), exclusive of tuition and related fees, direct expenditures for equipment, and subawards in excess of $25,000. (With the exception of the American University of Beirut and the World Health Organization, which are eligible for full F&A cost reimbursement).

Awards to domestic organizations with a foreign or international consortium participant may include 8 percent of MTDC, exclusive of tuition and related fees, direct expenditures for equipment, and subawards in excess of $25,000.

All instructions in the SF424 (R&R) Application Guide PHS 398 Cover Page Supplement All instructions in the SF424 (R&R) Application Guide All instructions in the SF424 (R&R) Application Guide must Sharing Plan : Individuals are required to comply with the instructions for the Resource Sharing Plans as provided in the SF424 (R&R) Application Guide, with the following modification: All applications, regardless of the amount of direct costs requested for any one year, should address a Data Sharing Plan.

Only limited Appendix materials are allowed.

Follow all instructions for the Appendix as described in the SF424 (R&R) Application PHS Human Subjects and Clinical Trials Information When involving human subjects research, clinical research, and/or clinical trials (and when applicable, clinical trials research experience) follow all instructions for the PHS Human Subjects and Clinical Trials Information form in the SF424 (R&R) Application Guide, with the following If you answered Yes to the question Are Human Subjects Involved?

on the R&R Other Project Information form, you must include at least one human subjects study record using the Study Record: PHS Human Subjects and Clinical Trials Information form or Delayed Record: PHS Human Subjects and Clinical Trials Information All instructions in the SF424 (R&R) Application Guide onset does NOT apply to a study that can be described but will not start immediately All instructions in the SF424 (R&R) Application Guide must PHS Assignment Request Form All instructions in the SF424 (R&R) Application Guide must 3.

Unique Entity Identifier and System for Award Management (SAM) See Part 1. Section III. 1 for information regarding the requirement for obtaining a unique entity identifier and for completing and maintaining active registrations in System for Award Management (SAM), NATO Commercial and Government Entity (NCAGE) Code (if applicable), eRA Commons, and Grants.

gov Part I. Overview Information contains information about Key Dates and times. Applicants are encouraged to submit applications before the due date to ensure they have time to make any application corrections that might be necessary for successful submission.

Organizations must submit applications to Grants. gov (the online portal to find and apply for grants across all Federal agencies). Applicants must then complete the submission process by tracking the status of the application in the eRA Commons , FDA’s electronic system for grants administration.

eRA Commons and Grants. gov systems check the application against many of the application instructions upon submission. Errors must be corrected and a changed/corrected application must be submitted to Grants.

gov on or before the application due date and time. If a Changed/Corrected application is submitted after the deadline, the application will be considered will not be accepted for this FOA. are responsible for viewing their application before the due date in the eRA Commons to ensure accurate and successful submission.

Information on the submission process and a definition of on-time submission are provided in the SF424 (R&R) Application Guide. 5. Intergovernmental Review This initiative is not subject to intergovernmental All FDA awards are subject to the terms and conditions, cost principles, and other considerations described in the HHS Grants Policy Statement and 45 CFR 75.

Pre-award costs are allowable only as described in the HHS Grants Policy Statement . Additional funding restrictions may be part of the Notice of 7. Other Submission Requirements Applications must be submitted electronically following the instructions described in the SF424 (R&R) Application Guide.

Paper applications will not be accepted. Applicants must complete all required registrations before the application due date. Section III.

Eligibility Information contains information about registration. For assistance with your electronic application or for more information on the electronic submission Apply Application Guide .

For assistance with application submission, contact the Application Submission Contacts in Section All PD(s)/PI(s) must include their eRA Commons ID in the Credential field of the Senior/Key Person Profile Component of the SF424(R&R) Application Package . Failure to register in the Commons and to include a valid PD/PI Commons ID in the credential field will prevent the successful submission of an electronic application to FDA.

See Section III of this FOA for information on registration requirements. The applicant organization must ensure that the DUNS number it provides on the application is the same number used in the organization s profile in the eRA Commons and for the System for Award Management. Additional information may be found in the SF424 (R&R) Application Guide.

for avoiding common errors. Upon receipt, applications will be evaluated for completeness and compliance with application instructions by the assigned Grants Management Specialist and responsiveness by components of participating organizations , FDA.

Applications that are incomplete, non-compliant and/or nonresponsive will Post Submission Materials Post-submission materials are those submitted after submission of the grant application but prior to objective review. They are not intended to correct oversights or errors discovered after submission of the application.

FDA accepts limited information between the time of initial submission of the application and the time of objective review. Applicants must contact the assigned Grants Management Specialist to receive approval, prior to submitting any post submission materials. Acceptance and/or rejection of any post submission materials is at the sole discretion of the FDA.

Any inquiries regarding post submission materials should be directed to the assigned Grants Management Section V. Application Review Information Only the review criteria described below will be considered Reviewers will consider each of the review criteria below in the determination of scientific merit. Does the project address an important problem or a critical barrier to progress in the field?

Is the prior research that serves as the key support for the proposed project rigorous? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

Investigator(s) (20 Points) Are the PD(s)/PI(s), collaborators, and other researchers well suited to the project? If Early Stage Investigators or those in the early stages of independent careers, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)?

If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions?

Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed? Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project?

Have the investigators included plans to address weaknesses in the rigor of prior research that serves as the key support for the proposed project? Have the investigators presented strategies to ensure a robust and unbiased approach, as appropriate for the work proposed? Are potential problems, alternative strategies, and benchmarks for success presented?

If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?

Have the investigators presented adequate plans to address relevant biological variables, such as sex, for studies in vertebrate animals If the project involves human subjects and/or clinical research, are the plans to address 1) the protection of human subjects from research 2) inclusion (or exclusion) of individuals on the basis of sex/gender, race, and ethnicity, as well as the inclusion or exclusion of individuals of all ages (including children and