Configure reliability assessment defaults
The response rate of a course evaluation survey is important to the credibility of the results. A low response rate may decrease the statistical power of the data collected and undermine the reliability of the results. It may also undermine the ability to generalize the results to the larger audience. This is further complicated by the fact that a low response rate can be indicative of a non-response bias. Thus, various studies examined the questions of validity, implementation mechanisms, adequacy of response rate, and sources of bias to properly interpret the quality of the information (for example, the effects of low response rates on the reliability and precision of the course evaluation measure).
Using the Reliability of the Results Assessment (hereafter referred to as Reliability Assessment) in Blue, institutions can set a certain level to help assess the reliability of the results, taking into account a response rate.
Reliability of the results in Blue is estimated based on the bound of the error of estimation in "Elementary Survey Sampling" by Richard L. Scheaffer et. al. In Blue, the bound of the error of estimation with the standard deviation (s) set to the constant 1.1 is called Reliability Assessment Score (RAS), as shown below.
In this formula, the RAS is in between 0 and 2.2. As the results approach 0 the confidence in the reliability of the results increases, whereas confidence in the reliability of the results decreases as the results near 2.2.
Default RAS Thresholds in Blue
Default RAS thresholds are set in Blue with the following assumptions:.
is considered to have a good number of responses
is considered to have a sufficient number of responses
is considered to have an insufficient number of responses
Institutions can change these default thresholds or set your own thresholds in Blue.
- To determine the ranges for your institution, create an export report that includes all of the subjects and add the Reliability Assessment Special field. This will provide the reliability assessment related output for all the subjects in a CSV file, which can then be used to baseline the reliability assessment ranges for your organization.
- Configure multiple Reliability Assessment levels to address the different needs of your schools, departments, etc.
InReport blocks