Candidate selection help to find fairest mechanism to use

Hello all,

Seeking some guidance to help me as I work out the score of a group of candidates for a set of roles I am hiring for.

They've had to undergo 2 exercises. One was where they were assessed during a networking session by a group of managers over the course of 90 minutes. In this time they could move freely between managers and each manager then recorded a score of 1-5 (5 being good) for each candidate.

Obviously each candidate spoke to a different numbers of managers and each to different managers introducing a couple of variables. One being the managers each scoring high and low in a subjective way, the other being the different number of assessments.

In the second exercise, a standard 2 on 1 interview, the candidates were asked up to 10 questions, and in each could score between 1 and 4 (4 being high). Again a different pair of managers for each candidate in most cases and there was no requirement for all 10 questions to be asked or for them to be asked in a set order. Again variables are the managers inconsistency but also that some may have only answered 4-5 questions whereas some may have managed 10.

So - finally we come to the question! For each of these, what do people think are the most appropriate way to score these? I've thought of straight averages but they are prone to outliers where a manager takes a dislike to the candidate (hopefully not!) and of mode, though this doesn't tend to give a precise enough score for me to rank the candidates very easily and also has issues whereby if someone scored say 4,4,4,1,1,1 in interview they get a mode of 4!

Guidance and suggestions most welcome all!