The Higher Education Randomized Controlled Trials (THE-RCT) – Supporting High Quality and Efficient Postsecondary RCTs and Open Science Practices

The Higher Education Randomized Controlled Trials (THE-RCT) – Supporting High Quality and Efficient Postsecondary RCTs and Open Science Practices is sponsored by Institute of Education Sciences (IES). Develops resources to support quality, efficiency, comparability, and transparency in postsecondary RCTs and open science practices.

The Higher Education Randomized Controlled Trials (THE-RCT) – Supporting High Quality and Efficient Postsecondary RCTs and Open Science Practices | IES The Higher Education Randomized Controlled Trials (THE-RCT) – Supporting High Quality and Efficient Postsecondary RCTs and Open Science Practices Statistical and Research Methodology in Education (08/01/2024 - 07/31/2027) Methodological Innovation The research team will develop resources to support the quality, efficiency, comparability, and transparency of research on the causal effects of postsecondary programs, policy, and practices at colleges and universities across the United States.

Over the last 20 years, researchers have conducted approximately 130 postsecondary RCTs that meet What Works Clearinghouse (WWC) standards without reservations. While these studies reflect WWC standards for designs that support causal inference, they do not necessarily follow best practices for open science that facilitate syntheses to draw lessons across bodies of evidence.

Many postsecondary RCTs that do not meet WWC standards, as well as some that do, are underpowered and contain inconsistencies in outcome reporting that make it difficult to draw conclusions across studies.

Moreover, the tools, materials, and computer code needed to implement high-quality RCTs are often re-invented by researchers and research organizations, missing opportunities to improve the quality and efficiency of implementing higher education RCTs.

In this methods project, researchers will develop a coherent and coordinated package of integrated resources for the design and implementation of rigorous randomized controlled trials (RCTs) of postsecondary education interventions.

The team will create, curate, and disseminate resources that will: (a) facilitate conducting more high-quality postsecondary RCTs, (b) reduce the cost of conducting such evaluations, (c) support the implementation of open science practices, and (d) increase consistency across studies in support of cross-study analyses.

These freely available resources will be end-user tested, and a network of 23 national experts will help create, curate, and vet these tools and materials, accelerate the formation of field norms, and use and disseminate these products. The research team will conduct analyses and build interfaces to make the resources accurate and accessible to users.

Research design and methods Researchers will assemble a list of key components for carrying out high quality RCTs, verified by the national network of experts. The agreed-upon list will serve as an umbrella for resource curation and development by the research team.

Within components requiring tool development, the research team will employ correlational analysis and iterative (continuous improvement) methods to populate the tools with current data and make them accessible to others. Within components where high-quality tools exist, the team will connect researchers with these tools and their supporting materials.

The project website will include accessible information about all tools that will make them usable by the broad field of postsecondary researchers, including college administrators, university-based researchers, and research firms.

This project's ongoing theory and empirical evidence-based resource development process will include updating earlier analyses with new data, enhancing analyses with new developments, and integrating resources.

Enhancements will include updated empirical benchmarks for interpreting study findings, updated design parameters for power/minimum detectable effects calculations, and updating state funding formulas for improved return-on-investment calculations.

Website resources and tools will be integrated by explicitly drawing connections between a tool designed to help researchers assess the likely available sample for a planned study, the minimum detectable effect (MDE) for a planned study, and where MDE falls within the distribution of effects from past evaluations. User Testing: Resources will be created, curated, reviewed, and vetted by the expert network.

They will review plans, test beta versions prior to release, and provide feedback on functionality and usability. The team will use network meetings to obtain additional feedback on products. Each product's development process will start with an initial full committee meeting, including one or more MDRC staff member and network experts who will also serve as end-user testers.

Initial committee meetings will clarify the goals of the product, gather information about existing resources that may be useful as a starting point for product development. Next, with the goals clarified and existing document review completed, one or two committee leads will draft the products. This process will be followed by a draft product review meeting with the full committee including network expert end-users.

This process will iterate the draft products two times or until the committee is satisfied with the products, whichever comes first. Once completed, the product will be shared electronically and/or at the annual network convening, for product review/vetting with the full expert network. Here, the broader network will have the opportunity to provide additional end-user feedback before the product is finalized.

People and institutions involved Commissioner of Education Research Co-principal investigator Products and publications ERIC Citations: Find available citations in ERIC for this award here .

An Intervention Return on Investment (ROI) Tool for Community Colleges The Higher Education Randomized Controlled Trial (THE-RCT) Project: Synthesizing Evidence from 15+ Years of RCTs in Postsecondary Education Planning Randomized Controlled Trials in Community Colleges Questions about this project? To answer additional questions about this project or provide feedback, please contact the program officer. Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer. Fact Sheet/Infographic/FAQ A Community of Practice on the Rural Educator Work... Fact Sheet/Infographic/FAQ Examining the English Language Arts Placement Syst...

Revenues and Expenditures for Public Elementary an... National Center for Education Statistics (NCES)