Reliability and validity are two concepts which are concurrently and interchangeably used to gauge the quality of a test, method or technique to evaluate and produce credible data.
While reliability is about the consistency (across different verticals and applications) of a test, validity is about calculating the “truth-value” (accuracy) of the test in the production of reliable and actionable data.
We, together with our technology partner, strive towards maintaining a healthy counteractive knowledge transaction and application process between these two concepts for each of our assessments.
|What does it reveal to us?||The test should be able to produce similar results when applied to similar contexts.||The test should be able to isolate, target and highlight the individual’s level of expertise in specific skill/skill sets.|
|How do we (generally) assess it?||By testing it against varying situations, different actors, and through a qualitative understanding of the question’s position as a building block in the overall structure of the test.||Benchmarking data, industry experts, maker-checker, and difficulty index.|
Validity forms the cornerstone of our process of procuring content, evaluating it and finally delivering it to the customer in a way that the latter has demanded it. We follow a strict validation process which includes, but is not limited to the following paradigms:
Let us understand the various forms of validity and how we implement them in our system.
|Construct validity||The adherence of a test structure to existing theory and knowledge of the skill being assessed.||If a particular skill/skill sets are to be tested, industry experts and SMEs co-relate the newly created test to how the skill has been tested till now and under what parameters to produce a comparative analysis.|
|Content validity||The extent to which the test covers all aspects of the concept being measured.||– Analysis of job description by subject matter experts.|
– Maker-checker process.
|Criterion validity||The extent to which the result of a test corresponds to other valid assessments of the same concept.||Benchmarking data.|
|Face validity||Face validity is the degree to which a test appears to be related to a specific skill/purpose in the judgment of non-experts such as test takers. Simply put, a test has face validity if its content looks relevant to the person taking the test.||– Candidate feedback.|
– Working with customers (hiring managers).
Reliability is, in essence, the measure of the consistency of a test across different paradigms. These forms of reliability testing exists in pre-employment assessment structures:
|Test-retest reliability||Replicability of tests and how test scores relate to each other when the same test is taken twice in a short amount of time.|
|Inter-rater reliability||Different observers attempt the same test and validate it accordingly. Their inter-relative judgments become the basis of inter-rater reliability.|
|Internal consistency||Similar to content validity insofar as how the test incorporates various elements cohesively in order to produce the desired result.|
A reliability coefficient is generally the procedure through which we can test how reliable a particular test is. We calculate the the co-relation between two different parameters (test performance and job performance) by a correlation coefficient (which ranges from -1.0 to +1.0). Most of our tests are within the range of +.75 – +.80.
Skilon ensures that each one of our tests go through the extensive processes of validity and reliability in order to produce data that is actionable, relevant and precise.