Picking Statistical Analysis - Does Funding Mitigate Impact of Poverty

#1
Hi everyone,

Thank you in advance for any help you can provide. I am new to the forum and hope to stick around for a while.

I am working on a research question that seems pretty simple:

Does increased school district level funding mitigate the impact of poverty on student achievement?

The measure of school district level funding = expenditures per student.
The measure of poverty = percentage of students receiving free or reduced priced lunch.
The measure of student achievement = student test scores (Grades 3, 5 and 8 ELA and Math assessments).

I indicated in my methodology, that was approved by my committee, that I would answer this question by using a multiple regression where the assessment scores are the dependent (outcome) variable and the independent variables are both the poverty and achievement variables.

Something about this doesn't feel right to me. Can anyone provide any insight as to whether this will work? (if so, how will the output show whether funding mitigates the impact?) If it won't work, what would?

Thanks!
 

noetsi

Fortran must die
#2
Poverty and the effect of public policy on it is not simple. There are likely thousands of articles on it in the last 50 years:p I am sure you will find, and perhaps have already, many of these articles.

How are test scores measured? On an interval scale, say 1-100, you can use linear regression. If on an ordinal or nominal scale you will need logistic regression. There are many variables other than the ones you listed which influence test score. One obvious one is parental involvement in children's education. You should look at adding some of these variables to your model. Religion, some argue also influences this (this is actually much debated in the education literature). School districts vary on many dimensions and students are nested inside schools. This may create problems for your analysis (because it violates the assumption of independence). Multilevel analysis has been applied to this type of issue specifically because of that type of issue I believe.

You talk of several different grades. Were you thinking of combining these somehow, or of measuring change between grades? Again multilevel analysis is applied here (in this case students are nested in time) and structural equation models are useful as well. Generally, I believe structural equation models are nearly always better than regression because they allow exploration of indirect effects including moderation much better than regression. But both multilevel and SEM are harder to do and require additional software.
 
#3
To be clear, I am VERY aware of other variables that impact student achievement. This RQ is setting the scene for a qualitative study that will delve deeper into the other variables.

I intend on looking at the grades separately and will thus be repeating this analysis eighteen times (2 tests, 3 grades, 3 different years of the study).

Test scores are interval scale.

So, it sounds like a linear regression will work fine. What part of the output will tell me whether the increased expenditures mitigated the impact of poverty?
 

noetsi

Fortran must die
#4
If the slope of the moderating variable is signficant you will know that it influenced the impact of the other IV (at least as long as there is some relationship between the two IV which you can tell by regressing one IV on the other). Regression shows partial slopes, that is the unique impact of an IV on the DV, so if the second IV had an impact on the DV and this explained variation overlapped with the other IV on the DV this overlap will be ignored in the calculated (partial) slope for the two IV. Knowing what specifically the moderating effect is, is more complex. I have seen manual calculations that gets at it, but not in print outs from standard software. You could compare the impact of the IV on the DV with and without the other IV of course.

I think you can get at moderating effects, and indirect effects, much better with SEM. Note that an entirely different issue is whether the two VI are interacting. This requires specification of an interaction term (if it is signficant than you have interaction). In this case you would need to generate simple effects to show the impact of one IV on the DV.