Dear all,
I have a sample size question that I'd like to consult your opinions on.

In our lab, we gave a particular measure to roughly 450 teachers in year one and roughly 400 teachers in year 2 to assess their teaching practices. Between year one and year 2, we have an overlap of 200 teachers who completed this measure twice and the rest were unique teachers who either completed the measure either in year one or in year 2. The reason that we gave the same measure to these individuals across the two years is to see whether the professional development we implemented improved teachers' teaching practice.

Now that we're interested in potentially testing all the way up to a 3rd order factor structure for this model, and we need as big of a sample size as possible, we want to know if it's possible to combine the unique teachers from year one and year 2, and choose either a year 1' or year 2's data for those teachers who completed the measure twice and validate the measure based on this sample. What objections/concerns do you think reviewers would have about our approach? If you have any suggestions of how best to do this, I'd appreciate it.

I was thinking one way of doing this is to simply test the first order factor structure for year one and year 2 separately(despite the fact that the same teachers are present in year one data and year two data) and then constrain the factor loadings for both years to be the same. If constraining the factor loadings does not produce a significant reduction in fit, that could justify that we could combine everyone's data together so that we can have the biggest sample size possible when it's time to validate the third order factor structure. Do you see any problems with this approach? But then, we'll have to decide which year we want to use for those teachers who completed the measure twice.

I sincerely appreciate your time and your thoughts on this.