Skilon Reliability and Validity Process

Reliability and validity are two concepts which are concurrently and interchangeably used to gauge the quality of a test, method or technique to evaluate and produce credible data.

While reliability is about the consistency (across different verticals and applications) of a test, validity is about calculating the “truth-value” (accuracy) of the test in the production of reliable and actionable data.

We, together with our technology partner, strive towards maintaining a healthy counteractive knowledge transaction and application process between these two concepts for each of our assessments.

What does it reveal to us?The test should be able to produce similar results when applied to similar contexts.The test should be able to isolate, target and highlight the individual’s level of expertise in specific skill/skill sets.
How do we (generally) assess it?By testing it against varying situations, different actors, and through a qualitative understanding of the question’s position as a building block in the overall structure of the test.Benchmarking data, industry experts, maker-checker, and difficulty index.

Validity forms the cornerstone of our process of procuring content, evaluating it and finally delivering it to the customer in a way that the latter has demanded it. We follow a strict validation process which includes, but is not limited to the following paradigms:

  • Construct
  • Content
  • Criterion
  • Face

Let us understand the various forms of validity and how we implement them in our system.

Construct validityThe adherence of a test structure to existing theory and knowledge of the skill being assessed.If a particular skill/skill sets are to be tested, industry experts and SMEs co-relate the newly created test to how the skill has been tested till now and under what parameters to produce a comparative analysis.
Content validityThe extent to which the test covers all aspects of the concept being measured.– Analysis of job description by subject matter experts.
– Maker-checker process.
Criterion validityThe extent to which the result of a test corresponds to other valid assessments of the same concept.Benchmarking data.
Face validityFace validity is the degree to which a test appears to be related to a specific skill/purpose in the judgment of non-experts such as test takers. Simply put, a test has face validity if its content looks relevant to the person taking the test.– Candidate feedback.
– Working with customers (hiring managers).

Reliability is, in essence, the measure of the consistency of a test across different paradigms. These forms of reliability testing exists in pre-employment assessment structures:

Test-retest reliabilityReplicability of tests and how test scores relate to each other when the same test is taken twice in a short amount of time.
Inter-rater reliabilityDifferent observers attempt the same test and validate it accordingly. Their inter-relative judgments become the basis of inter-rater reliability.
Internal consistencySimilar to content validity insofar as how the test incorporates various elements cohesively in order to produce the desired result.

A reliability coefficient is generally the procedure through which we can test how reliable a particular test is. We calculate the the co-relation between two different parameters (test performance and job performance) by a correlation coefficient (which ranges from -1.0 to +1.0). Most of our tests are within the range of +.75 – +.80.

Skilon ensures that each one of our tests go through the extensive processes of validity and reliability in order to produce data that is actionable, relevant and precise.

Making your skills shine

Our core mission is to make your skills shine. Take our tests and publish your proven skills to reach recruiters and headhunters easily and quickly. We are here to diversify the workforce by shedding the light on qualified candidates and what they know rather than what they look like.