Reliability and validity are two concepts which are concurrently and interchangeably used to gauge the quality of a test, method or technique to evaluate and produce credible data.
While reliability is about the consistency (across different verticals and applications) of a test, validity is about calculating the “truth-value” (accuracy) of the test in the production of reliable and actionable data.
We, together with our technology partner, strive towards maintaining a healthy counteractive knowledge transaction and application process between these two concepts for each of our assessments.
| RELIABILITY | VALIDITY | |
|---|---|---|
| What does it reveal to us? | The test should be able to produce similar results when applied to similar contexts. | The test should be able to isolate, target and highlight the individual’s level of expertise in specific skill/skill sets. |
| How do we (generally) assess it? | By testing it against varying situations, different actors, and through a qualitative understanding of the question’s position as a building block in the overall structure of the test. | Benchmarking data, industry experts, maker-checker, and difficulty index. |
Validity forms the cornerstone of our process of procuring content, evaluating it and finally delivering it to the customer in a way that the latter has demanded it. We follow a strict validation process which includes, but is not limited to the following paradigms:
- Construct
- Content
- Criterion
- Face
Let us understand the various forms of validity and how we implement them in our system.
| Construct validity | The adherence of a test structure to existing theory and knowledge of the skill being assessed. | If a particular skill/skill sets are to be tested, industry experts and SMEs co-relate the newly created test to how the skill has been tested till now and under what parameters to produce a comparative analysis. |
| Content validity | The extent to which the test covers all aspects of the concept being measured. | – Analysis of job description by subject matter experts. – Maker-checker process. |
| Criterion validity | The extent to which the result of a test corresponds to other valid assessments of the same concept. | Benchmarking data. |
| Face validity | Face validity is the degree to which a test appears to be related to a specific skill/purpose in the judgment of non-experts such as test takers. Simply put, a test has face validity if its content looks relevant to the person taking the test. | – Candidate feedback. – Working with customers (hiring managers). |
Reliability is, in essence, the measure of the consistency of a test across different paradigms. These forms of reliability testing exists in pre-employment assessment structures:
| Test-retest reliability | Replicability of tests and how test scores relate to each other when the same test is taken twice in a short amount of time. |
| Inter-rater reliability | Different observers attempt the same test and validate it accordingly. Their inter-relative judgments become the basis of inter-rater reliability. |
| Internal consistency | Similar to content validity insofar as how the test incorporates various elements cohesively in order to produce the desired result. |
A reliability coefficient is generally the procedure through which we can test how reliable a particular test is. We calculate the the co-relation between two different parameters (test performance and job performance) by a correlation coefficient (which ranges from -1.0 to +1.0). Most of our tests are within the range of +.75 – +.80.
Skilon ensures that each one of our tests go through the extensive processes of validity and reliability in order to produce data that is actionable, relevant and precise.
