Add a reliability assessment report block

The response rate of a course evaluation survey is important to the credibility of the results. A low response rate may decrease the statistical power of the data collected and undermine the reliability of the results. It may also undermine the ability to generalize the results to the larger audience. This is further complicated by the fact that a low response rate can be indicative of a non-response bias. Thus, various studies examined the questions of validity, implementation mechanisms, adequacy of response rate, and sources of bias to properly interpret the quality of the information (for example, the effects of low response rates on the reliability and precision of the course evaluation measure).

Using the Reliability of the Results Assessment (hereafter referred to as Reliability Assessment) in Blue, institutions can set a certain level to help assess the reliability of the results, taking into account a response rate.

Reliability of the results in Blue is estimated based on the bound of the error of estimation in "Elementary Survey Sampling" by Richard L. Scheaffer et. al. In Blue, the bound of the error of estimation with the standard deviation (s) set to the constant 1.1 is called Reliability Assessment Score (RAS), as shown below.

In this formula, the RAS is in between 0 and 2.2. As the results approach 0 the confidence in the reliability of the results increases, whereas confidence in the reliability of the results decreases as the results near 2.2.

Default RAS Thresholds in Blue

Default RAS thresholds are set in Blue with the following assumptions:

• is considered to have a good number of responses
• is considered to have a sufficient number of responses
• is considered to have an insufficient number of responses

Institutions can change these default thresholds or set your own thresholds in Blue.