Fact Sheet: Assessment Validation

Assessment validation fact sheet - Print version - PDF (524KB) 

This Training Accreditation Council (TAC) fact sheet provides guidance to Registered Training Organisations in determining that their assessment system is meeting the requirements of the Standards for Registered Training Organisations (RTOs) 2015 for Clause 1.8, and for Clauses 1.9, 1.10 and 1.11.

This Fact Sheet does not fully address the requirements of Clause 1.25 and Schedule 2 of the Standards for RTOs which relate to the independent validation of training and assessment qualifications. Further information on these requirements can be found in the Fact Sheet: Delivery of Training and Assessment Qualifications and further information on TAC’s Assessment workshop series is available via the TAC webinar recordings and resources webpage. 

Assessment validation explained:

Clause 1.8a requires that the RTO’s assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses, and Clause 1.8b requires RTOs to ensure that evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment). These requirements must be met and demonstrated by all the assessment policies, procedures, materials and tools of the RTO. Clause 1.8 relates primarily to the development (or purchase) of the RTO’s assessment resources.

Assessment validation is emphasised in the Standards for RTOs, and RTOs are required to:

• develop and implement a comprehensive plan for the ongoing systematic validation of assessment that includes all training products on the RTO’s scope of delivery (Clause 1.9);
• validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five year cycle (Clause 1.10); and
• ensure that validation is conducted by one or more suitably qualified persons (Clause 1.11).

These clauses relate primarily to the actual delivery and outcomes of the RTO’s assessment systems, including the performance of the RTO’s assessors.

Assessment validation should answer the question:

‘Does the assessment practices meet the requirements of the training package, and the Standards for RTOs and are assessment judgements being consistently applied so that certification issued is accurate and credible?’

Assessment validation needs to be undertaken across all stages of the assessment process of design, the application of evidence-gathering tools, and the assessment judgement.

The validity of the assessment system design process is addressed under Clause 1.8 and applies to all assessment resources. Validation of the application of the assessment system is addressed through Clauses 1.9 - 1.11 and is conducted with a sample of assessments.

The overall outcome of assessment validation is to ensure:

• assessments are valid and meet the training package or accredited course requirements;
• the principles of fairness, flexibility, validity and reliability are met;
• the evidence used to make a decision about competence is valid, sufficient, authentic and current;
• assessors are familiar with the outcomes being assessed;
• assessors reach accurate and consistent decisions that inform assessment outcomes; and
• potential improvements to assessment practices are identified and acted upon.

Who can be involved in validation of assessment?

This validation process can be undertaken by a team or individual approach as long as individually or collectively, the team or individual meets the following requirements outlined in Clause 1.11.

1. vocational competencies and current industry skills relevant to the assessment being validated;
2. current knowledge and skills in vocational teaching and learning; and
3. the training and assessment credential specified in Item 2 or Item 5 of Schedule. 1

Previous versions of the Standards did not allow assessors who were directly involved in the particular instance of delivery and assessment be involved in the validation process. However, Clause 1.11b now states that systematic validation:

‘may be undertaken by persons involved in the delivery and assessment of the training product being validated provided they are not solely responsible for determining the outcomes of validation.’

This expands the categories of individuals who are permitted to undertake validation for the purposes of Clause 1.9, thereby reducing the administrative burden for RTOs, whilst ensuring independent oversight of validation outcomes.

The validity of the design or purchase of assessment resources

The design or purchase of assessment tools is considered part of the RTO’s assessment system under Clause 1.8.

All assessment resources, whether purchased or designed internally, must be validated prior to use in accordance with the Principles of Assessment and the Rules of Evidence. A mapping document where assessment tools are mapped to the specific requirements of the unit of competency is typical evidence that an RTO has checked the validity of the assessment tools and would generally be considered partial evidence of the validity of assessment if it was accurate.

The purpose of assessment validation at the design or purchase stage is to verify if the assessment resources:

• meet the principles of fairness, flexibility, validity and reliability;
• meet all the rules of evidence of validity, sufficiency, currency and authenticity;
• provide clear instructions for evidence gatherers and candidates; and
• provide transparent and defensible information to RTOs in relation to their competency decisions and awarding of nationally recognised qualifications and/or statements of attainment.

Where to Start:

Assessment validation schedule

It is important RTOs have a clear understanding of the terms being used before they commence developing assessment validation plans and schedules under Clauses 1.9 -1.11. For example, in the context of the Standards for RTOs:

1. What is a training product?
2. What is the intent of assessment validation?
3. What is a statistically valid sample?

The answers to all these questions are detailed in the Glossary of the Standards for RTOs.

Clause 1.10 of the Standards for RTOs states:

“…each training product is validated at least once every five years, with a least 50% of products validated within the first three years of each five year cycle…”

The intent of this requirement is that over five years the assessment practices and judgements of all training products on an RTO’s scope of delivery must be systemically validated including those where there have been no enrolments.

As noted earlier, the intent of assessment validation is to establish:

‘Is the assessment process actually doing what it is supposed to be doing?’

This question needs to be answered across both stages of the application of the assessment process including:

• the gathering of evidence using the assessment tools; and
• the assessment judgement based on the evidence.

These stages will most likely be a starting point for informing the RTO’s assessment validation plan.

Assessment validation plan

This stage of the assessment validation process is a quality process that evaluates if the:

• assessors reached accurate and consistent decisions that informed nationally recognised training outcomes;
• Rules of Evidence have been met through the collection of valid, sufficient, authentic and current evidence;
• competency decisions are valid, fair and reliable in accordance with the Principles of Assessment; and
• assessment validation outcomes and actions are used to inform continuous improvement of assessment.

When validating a whole qualification or a skill set the assessment practices and judgements of a minimum of at least two units of competency must be validated. The two units selected need to be typical of the qualification or skill set.

It is anticipated that RTOs will use a risk analysis approach considering the complexity of the qualification and the risks associated with each unit of competency to determine the actual number of units of competency that will be validated. If the risk analysis highlights the presence of several risk factors that may compromise the quality of the assessment, or your validation indicates that assessment processes are not valid then more units should be validated.

When validating stand-alone units of competency, samples of assessment judgements for the full unit must be validated.

Statistically valid sample

In the Glossary of the Standards for RTOs a statistically valid sample:

‘… means for the purposes of these Standards, a random sample of appropriate size is selected to enable confidence that the result is sufficiently accurate to be accepted as representative of the total population of assessments being validated.’

The method an RTO uses to determine their assessment validation sample is an individual RTO decision. An RTO must validate a random sample of assessments. The selection of a completely random sample has many strengths, but the critical aspect of any sample is that it must be sufficient to ensure consistency of assessment decisions across each training product.

It is suggested that a ‘stratified random sample’ be used. This involves identifying distinct groups of candidates (such as campus based, workplace based, RPL, etc) and randomly selecting an appropriate number of students from each group for close analysis. Over time, all assessors, assessment methods and tools should be included in the sample. It is good practice to document all sampling strategies in a written internal verification procedure.

There are several key features that should be considered in selecting an assessment validation sample including (but not limited) to:

• student enrolment numbers in each training product;
• scope of delivery;
• feedback from clients, trainers and students;
• complaints received – formal or informal;
• completion rates;
• risk level of a training product;
• assessment mode (Simulation, workplace, RPL);
• introduction of new assessment activities/ tools;
• introduction of new training products;
• new assessors;
• third party arrangements; and
• previous issues or problems identified from assessment validation.

RTOs should note that there is no ‘one right way’ for the conducting of systematic assessment validation of assessment practices, individual RTOs are best placed to establish an appropriate plan and schedule.

Validating the gathering of assessment evidence

An RTO’s assessment validation plan will include monitoring of the use of assessment tools by evidence-gatherers to ensure they are implemented appropriately and are being used in accordance with the Rules of Evidence.

The application stage of assessment validation may also verify those who are gathering evidence are familiar with assessment tools and the outcomes they are assessing.

This may be done by observing the use of assessment tools to verify that the evidence being gathered is an accurate reflection of the relevant unit requirements and the candidate’s performance.

The monitoring of the validity of the application of assessment tools should be ongoing through the assessment process, not just at the end. Indications of problems may be through student feedback, appeals and complaints. This allows modifications to the evidence-gathering process before final assessment judgements are made. Identifying problems at an early stage will also allow support or training to be provided to evidence gatherers. 

Validating assessment judgements

Having validated the processes of gathering evidence, the validation should now focus on how the evidence is used to make the assessment judgement.

This can be done by establishing that there is a direct link between the evidence observed and the judgement of competence.

In particular, the validation could confirm that the Principles of Assessment have been met, including:

• all unit requirements have been observed before competence is awarded;
• different assessors using the same evidence would make consistent reliable judgements;
• all available evidence has been considered before making the judgement; and
• the candidate has had the opportunity to demonstrate all unit requirements.

The assessment validation team or person should have access to the same evidence in the sample that the assessor used to make the original assessment judgment. This is so the validator/s can reliably ensure that the assessment practice and judgement are valid.

Recording assessment validation outcomes

The outcomes of assessment validation may lead to recommendations for improvement to assessment tools, assessment processes or assessment judgements.

Your RTO must have a process to maintain records of assessment validation activities. You should retain evidence of:

• the person/people leading and participating in the assessment validation process including their qualifications, industry skills and VET knowledge;
• the training product and selected units of competency;
• the evidence-gathering tools for those units;
• the assessment validation tools used;
• all assessment samples considered; and
• the assessment validation outcomes.

The assessment validation plan must clearly show how you will document and act on assessment validation outcomes.

The validation plan must demonstrate how the assessment validation decisions will inform the continuous improvement of assessment practices and improvements. These improvements should be introduced across all training products. If changes and improvements are made to an assessment tool, this tool should be reviewed again prior to implementation.

An RTO with a well-designed assessment system and accompanying validation process that have been fully implemented will be able to demonstrate that their assessment judgements:

• are valid;
• align with the requirements of the unit of competency or module, and
• comply with the Standards for RTOs.

What are the auditors looking for?

During an audit the auditors are looking for evidence that you have established a system that assures the quality of assessment in your RTO. This will include:

• having processes and tools that enable validation at all stages of the lifecycle of a training product in your organisation;
• having a plan for validation that is implemented successfully in your organisation;
• being able to demonstrate that you have managed who participates in the validation process;
• being able to demonstrate that your assessors have made quality assessment decisions; and
• your assessment practice has improved because of validation activities