Question
Answered step-by-step
saoliphant2
Post a brief outline of the case study and consider the conclusion…
Post a brief outline of the case study and consider the conclusion that “the vocational rehabilitation intervention program may be effective at promoting full-time employment.”
What statistical information shows whether the program was effective (or not)?
Review the factors that limit the internal validity of a study (history, maturation, testing, instrumentation, statistical regression, selection bias, and attrition).
Select and explain which of these factors might limit the ability to draw conclusions regarding cause-and-effect relationships.
Dudley, J. R. (2020). Social work evaluation: Enhancing what we do (3rd ed.). Oxford University Press.
Chapter 10, “Analyzing Evaluation Data” (pp. 255-275)
Flannelly, K. J., Flannelly, L. T., & Jankowski, K. R. B. (2018). Threats to the internal validity of experimental and quasi-experimental research in healthcare Links to an external site.. Journal of Health Care Chaplaincy, 24(3), 107-130. https://doi.org/10.1080/08854726.2017.1421019
Chi-Square SOCW 6311
Molly, an administrator with a regional organization that advocates for alternatives to long-term prison sentences for nonviolent offenders, asked a team of researchers to conduct an outcome evaluation of a new vocational rehabilitation program for recently paroled prison inmates. The primary goal of the program is to promote full-time employment among its participants. To evaluate the program, the evaluators decided to use a quasi-experimental research design. The program enrolled 30 individuals to participate in the new program. Additionally, there was a waiting list of 30 other participants who planned to enroll after the first group completed the program. After the first group of 30 participants completed the vocational program (the “intervention” group), the researchers compared those participants’ levels of employment with the 30 on the waiting list (the “comparison” group). The research question for the study is: Is there a relationship between the independent variable, treatment, and the dependent variable, employment level? In other words, is there a difference in the number of participants who are not employed, employed parttime, and employed full-time in the program and the control group (i.e., waitlist group)? Data Collection and Analysis In order to collect data on employment levels, the probation officers for each of the 60 people in the sample (those in both the intervention and comparison groups) completed a short survey on the status of each client in the sample. The survey contained demographic questions that included an item that inquired about the employment level of the client. This was measured through variables identified as none, part-time, or fulltime. A hard copy of the survey was mailed to each probation officer and a stamped, self-addressed envelope was provided for return of the survey to the researchers. After the surveys were returned, the researchers entered the data into an SPSS program for statistical analysis. Because both the independent variable (participation in the vocational rehabilitation program) and dependent variable (employment outcome) used nominal/categorical measurement, the bivariate statistic selected to compare the outcome of the two groups was the Pearson chi-square. The chi-square test for independence is used to determine whether there is a relationship between the two variables that are categorical in the level of measurement. In this case, the variables are: employment level and treatment condition. It tests whether there is a difference between groups. The hypotheses are: H0 (The null hypothesis): There is no difference in the proportions of individuals in the three employment categories between the treatment group and the waitlist group. In other words, the frequency distribution for variable 2 (employment) has the same proportions for both categories of variable 1 (program participation). Note: It is the null hypothesis that is actually tested by the statistic. A chi-square statistic that is found to be statistically significant (p < .05) indicates that we can reject the null hypothesis (understanding that there is less than a 5% chance that the relationship between the variables is due to chance). H1 (The alternative hypothesis): There is a difference in the proportions of individuals in the three employment categories between the treatment group and the waitlist group. Note: The alternative hypothesis states that there is a difference. It would allow us to say that it appears that the treatment (vocational rehab program) is effective in increasing the employment status of participants. Results After all of the information was entered into the SPSS program, the following output charts were generated: Table 1. Case Processing Summary Cases Valid Missing Total N Percent N Percent N Percent Program Participation *Employment 59 98.3% 1 1.7% 60 100.0% The first table, titled "Case Processing Summary," provided the sample size (N = 59). Information for one of the 60 participants was not available, while the information was collected for all of the other 59 participants. Table 2. Program Participation *Employment Cross Tabulation Employment None Part-Time Full-Time Total Program Participation Intervention Group Count % within Program Participation 5 16.7% 7 23.3% 18 60.0% 30 100.0% Comparison Group Count % within Program Participation 16 55.2% 7 24.1% 6 20.7% 29 100.0% Total Count % within 21 35.6% 14 23.7% 24 40.7% 59 100.0% The second table, "Program Participation Employment Cross Tabulation," provided the frequency table, which showed that among participants in the intervention group, 18 or 60% were found to be employed full time, while 7 or 23% were found to be employed part time, and 5 or 17% were unemployed. The corresponding numbers for the comparison group (parolees who had not yet enrolled in the program but were on the waiting list for admission) showed that only 6 or 21% were employed full-time, while 7 or 24% were employed part time, and 16 or 55% were unemployed. The third table, which provided the outcome of the Pearson chi-square test, found that the difference between the intervention and comparison groups were highly significant, with a p value of .003, which is significantly beyond the usual alpha-level of .05 that most researchers use to establish significance. Discussion These results indicate that the vocational rehabilitation intervention program may be effective at promoting full-time employment among recently paroled inmates. However, there are multiple limitations to this study, including that 1) no random assignment was used, and 2) it is possible that differences between the groups were due to preexisting differences among the participants (such as selection bias). Potential future studies could include a matched comparison group or, if possible, a control group. In addition, future studies should assess not only whether or not a recently paroled individual obtains employment but also the degree to which he or she is able to maintain employment, earn a living wage, and satisfy other conditions of probation.
