Are you assuming the covariance matrix looks like (ie the observations have constant variance and are independent)?

If so and if you only have an intercept and a slope and you're using the residuals as your stand in for the errors (so instead of you use ) it is typical to use

Also note that to get the population covariance matrix you actually need E[ee'].