I am evaluating the effects that various factors may or may not have on the output of a process. Each of the factors has a different level.

[http://i.imgur.com/8UhG4ri.png]Here[/http://i.imgur.com/8UhG4ri.png] is the general layout in tabular form

My research group wants to see if any of the overall factors outputs a significantly different result (basically testing the robustness of the system). My original thoughts were to just use a basic multi-factor anova test to evaluate the results.

[http://i.imgur.com/DTU1dwZ.png]However[/http://i.imgur.com/DTU1dwZ.png], here is my experimental setup.

I've done n runs with a certain factor/level of interest (ie A1) then swapped out (A1) and did n runs with A2 etc. After refreshing my knowledge of stats, I've noticed that in order to properly perform an anova test, you need to perform runs with every combination of factors. Given the time constraints of the project, this was not feasible. My problem is that (for example) when I tested factor E2 (setup would be A1,B1,C1,D1,E2), the resultant mean was significantly higher than other tests. Therefore, we'd say E2 had a significant effect on the system, but because A1,B1,C1, and D1 were tested along with E2, there overall means are also effected.

Any guidance would be greatly appreciated, I'd like to evaluate the effects of each factor change independantly without A1,B1,C1, and D1 being effected by other tests.