Some assert that noise (i.e., unwanted variance) is the most neglected yet most important source of error in judgment. We suggest that this problem was discovered nearly 100 years ago in the area of personnel selection and that a century of selection research has shown that noise can be demonstrably reduced by structuring the process (i.e., decomposing the component parts, agreeing on standards, and applying those standards consistently) and by aggregating judgments independently. Algorithms can aid significantly in this process but are often confused with methods that, in their current form, can substantially increase noise in judgment (e.g., artificial intelligence and machine learning).
