Test to identify bias in staff recording data - some guidence please

Hi, First post on this forum. Have tried to search for the correct answer in this forum, but have not found it (perhaps I'm not searching correctly)

I am involved in a project where staff observe items and record their observations on an input device (iPads in this case). The data set is quite large (+750,000 records). Staff record usage (0%,25%,50%,75%,100%) using radio buttons, with about 8 items per observation.

For Quality Assurance purposes I need to identify 2 bias potential issues.

1 - Is the average of the recordings of a specific staff member statistically invalid because it is different from the average of all the other staff? (they don't know how to gauge the items and need training) - I assume that a simple ANOVA test will do this.

2 - Is there a propensity to record all the items on a record with the same value (e.g. They observe usage of the 8 items, which should all be slightly different in theory, as exactly the same for all the items on that observation, as it is easier to do so than differentiate each item individually.) It implies that they are not showing due care, and not recording correctly

I would appreciate it if someone could confirm approach for #1 and point me in the right direction for #2 as I am very rusty with my stats, I probably know enough to get myself into trouble....

Thanks in advance, Peter
Last edited: