Learning analytics

#1
Hi all. I am a newbie to data analysis. I am an online learning facilitator for an online seminar-based program for local high school students. Since it is a 100% online program, there are no face-to-face sessions. They have to complete online short quizzes (1 question per quiz, 2 to 3 quizzes for each seminar) and long quizzes (one long quiz for one seminar). They can also engage in discussion and post questions of their own interests and other students can join their discussions. Besides, I also set up some discussion questions for the students to participate.

I would like to look into how to measure the engagement of online activities of students. Should I be looking for followings as measurements of the online engagement?

1. Correlation between the final score of a student and his/her total hitcounts in the learning management system
2. Correlation between the final score of a student and the total time spent on watching the online seminars
3. Correlation between the final score of a student and how many times he/she repeated watching the same videos
4. Correlation between the final score of a student and the number of his/her posts in the online forum

Any help would be greatly appreciated! Thanks!

Ronald
 
#2
all those make reasonable predictors in a regression.

I do have a few concerns that stem from similar things I have worked on-
1. your sample size might be too small to detect a meaningful effect size
2. you need to control for a few covariates. For one, the best covariate to control for is prior GPA. That will sort out quite a few issues that might creep into your analysis.
3. you need to decide on an engagement variable or else you might deal with multicollinearity.

There are ways to deal with this issue, if you have enough students, you can create a component called "engagement" and then test that against their score with prior gpa as a covariate. That would probably make for a reasonable and interesting result.

What i mean by a component is to use principal component analysis with your 4 measures of engagement to create what is like a "summary" statistic based on their correlations with each other.
 
#3
Thank you very much for your help, the42up. I will need to look into your suggestions as I am not very familiar with statistics.

I think I have enough students..there are over 1000 students who are active on the platform in which about more than 100 of them participated online discussions. However, since the program is an extra-curricular activity, I don't have access to the GPA of students.

I agree that carrying out a PCA caclulation with the 4 measures is a brilliant idea to obtain the "summary" statistics. Thanks a lot!