When purchasing assessments and learner resources, organisations should be careful

When purchasing assessments and learner resources, organisations should be careful

Assessments and learner resources are an important part of any organisation's educational offering. When selecting and using assessments, organisations should be aware of a number of factors that can impact their effectiveness. By taking the time to carefully consider these factors, organisations can ensure that they are using assessments in the most appropriate and effective way possible.

When purchasing assessments and learner resources, organisations should be careful about the following things:

1. Make sure the assessments and learner resources meet training package requirements 

This step necessitates you to concentrate on how the assessment materials fit the requirements of the training package. This is the step when you understand what competence in this particular unit of competency will look like.

Focus points include understanding of:

• What is the AQF level where the unit of competency will be used?
• What is the unit descriptor/application of the unit saying about work activities included in the unit of competency?
• What are the prerequisite or corequisite requirements related to the unit of competency?
• What level of skill is required for this unit according to where (which qualification) the unit of competency will be used
• What are the elements, performance criteria, range of conditions, foundation skills, knowledge evidence, performance evidence, assessment conditions

Read the assessment conditions and foundation skills: What are the conditions under which this work activity should be conducted

• Are there any other specific requirements applicable to this unit of competency?

Before moving on to practical task activities, the learner must first demonstrate that he or she understands the subject through demonstration of knowledge.

KNOWLEDGE – you need to have knowledge before you can perform

Look over the requirements for the training package and have a close look at the knowledge evidence to see if it says once is sufficient. If it does not state that, it implies that you must address each of the knowledge evidence criteria at least twice. We can address the knowledge evidence requirements using a variety of activities such as questions and answers, case studies, report writing, and other knowledge-based assessment methods.

PERFORMANCE – means that you have to do something

Then it’s time to look at performance criteria and performance evidence, and once again, pay attention to whether or not there are instructions on how many times this should be addressed. If this is not the case, each performance criteria and the performance evidence must be addressed in the assessment tasks and activities at least twice, if not more, utilising a variety of assessment methods and activities such as projects, portfolios, practical task activities, workplace tasks and observations and so on.

Focus on the action verbs and action keywords

Focus on all action verbs and action keywords included in the training package when developing your assessment resources. Each and every action verb and keyword must be addressed through the assessment resources. 

2. Ensure that the assessments meet the principles of assessment and rules of evidence 

You must ensure that the assessment resources meet the principles of assessment and rules of evidence. In case you are not sure how read these articles: Principles Of Assessment https://vetsector.com.au/principles-of-assessment/  and  Rules Of Evidence https://vetsector.com.au/rules-of-evidence/ 

3. Choose assessments and learner resources that are appropriate for the intended audience.

To ensure their usefulness and appropriateness, assessments and learner resources must be appropriate for the intended audience.

4. Be aware of the costs associated with assessment development and delivery.

You must consider all the costs associated with the assessment and learner resources, the cost of contextualising and customisation to your learner cohort and delivery mode to cost of formatting and templating the documents according to your style guide. 

5. Ask for a sample before you purchase the training product! 

In order to ensure that the training product you intend to purchase meets all your quality requirements and expectations, you should request a sample. 

6. The assessment resources include a comprehensive mapping tool 

A comprehensive mapping document is required to ensure all training package criteria has been addressed appropriately and comprehensively. Mapping is a cross-referencing activity where each component of the unit of competency is cross-referenced to one or more assessment criteria or questions in the assessment activities and tasks. Mapping is more a content validity process and not a process validity process.

Refer to the following articles for more information

Mapping document of assessment resources – do you need one? https://caqa.com.au/2021/10/04/mapping-document-of-assessment-resources-do-you-need-one/ 

7. Focus on evidence collection and assessment methods 

Focus on evidence collection and assessment methods after ensuring that the assessment resources meet the training package requirements. The focus points should include:

• What are the assessment methods selected for evidence collection?
• Are these suitable and appropriate for evidence collection?
• What are the other methods that may be used for evidence collection?
• Where and how should evidence be collected?
• What resources are required for evidence collection?

Foundation skills, assessment conditions, performance evidence, performance criteria and knowledge evidence should be taken into consideration when designing the evidence collection and assessment methods.

The evidence collection and assessment methods should change according to the AQF level where the units of competency will be used. For example, for a Certificate II, III level true or false, match the following statements with, multiple-choice questions, fill in the blanks might be appropriate but for Certificate IV and Diploma short answer questions, closed book, time-limited exams, contrast and separate, and other assessment methods could be used. We have included an AQF summary for you to understand how each AQF level requires a different set of requirements.

AQF Level	Summary	Qualifications	Purpose of this Qualification
1	Graduates at this level will have knowledge and skills for initial work, community involvement and/or further learning	Certificate I	basic functional knowledge and skills to undertake work, further learning and community involvement.
2	Graduates at this level will have knowledge and skills for work in a defined context and/or further learning	Certificate II	qualify individuals to undertake mainly routine work and as a pathway to further learning.
3	Graduates at this level will have theoretical and practical knowledge and skills for work and/or further learning	Certificate III	to qualify individuals who apply a broad range of knowledge and skills in varied contexts to undertake skilled work and as a pathway for further learning.
4	Graduates at this level will have theoretical and practical knowledge and skills for specialised and/or skilled work and/or further learning	Certificate IV	to qualify individuals who apply a broad range of specialised knowledge and skills in varied contexts to undertake skilled work and as a pathway for further learning.
5	Graduates at this level will have specialised knowledge and skills for skilled/paraprofessional work and/or further learning	Diploma	to qualify individuals who apply integrated technical and theoretical concepts in a broad range of contexts to undertake advanced skilled or paraprofessional work and as a pathway for further learning.
6	Graduates at this level will have broad knowledge and skills for paraprofessional/highly skilled work and/or further learning	Advanced Diploma Associate Degree	to qualify individuals who apply specialised knowledge in a range of contexts to undertake advanced skilled or paraprofessional work and as a pathway for further learning.
7	Graduates at this level will have broad and coherent knowledge and skills for professional work and/or further learning	Bachelor Degree	to qualify individuals who apply a broad and coherent body of knowledge in a range of contexts to undertake professional work and as a pathway for further learning.
8	Graduates at this level will have advanced knowledge and skills for professional highly skilled work and/or further learning	Bachelor Honours Degree Graduate and Vocational Graduate Certificate Graduate and Vocational Graduate Diploma	to qualify individuals who apply a body of knowledge in a specific context or range of contexts to undertake professional or highly skilled work and as a pathway for research and further learning.
9	Graduates at this level will have specialised knowledge and skills for research, and/or professional practice and/or further learning	Masters Degree	to qualify individuals who apply an advanced body of knowledge in a range of contexts for professional practice and as a pathway for further learning.
10	Graduates at this level will have a systematic and critical understanding of a complex field of learning and specialised research skills for the advancement of learning and/or for professional practice	Doctoral Degree	to qualify individuals who apply a substantial body of knowledge to research, investigate and develop new knowledge, in one or more fields of investigation, scholarship or professional practice.

You must look if the assessment methods accurately and properly describe how many questions students must do correctly to be deemed satisfactory in the assessment task or activity and then check mapping to ensure your recommendation does not compromise the integrity of the assessment.

Always remember that each of the evidence collection and assessment methods must flesh out the details related to the assessment activities and tasks such as what, why, where, how, and when something must occur.

8. Check the content for validity and reliability

Ensure all content is complete, error-free, and plagiarism and copyright issues free, you also need to ensure that:

• Assessment resources have sufficient and clear information regarding what, when, how, where, why for your assessment template and all assessment tasks and activities.
• Assessment resources have robust benchmarking and/or trainers’ guidance.
• Assessment resources are allowing the trainer/assessor to assess the skills and knowledge of students through different assessment tasks over a period of time to ensure consistency and sufficiency.
• Each and every question and assessment task has very clear guidelines around what is expected from the students in terms of both quantity and quality.
• You have customised the off-the-shelf resources according to your RTO needs and requirements and not using them as-is.
• Your assessment resources are written by industry experts with subject matter experts and are industry-relevant and current.
• Your assessment resources address all requirements of the training packaging rules
• Your assessment resources have detailed and valid performance checklists/observation checklists for assessing and observing the students before, during and after any skill assessment activity or workplace task
• Your trainers and assessors gather sufficient, valid evidence for competency assessment
• Your organisation offers appropriate simulated environments for conducting assessments
• The authenticity of assessment, particularly in distance and online delivery is established and maintained
• The context and conditions of assessment. For example, an assessment tool is developed to cater for a particular language, literacy and numeracy requirements, the learner’s workplace experience or other learner needs that require reasonable adjustment.
• The context of the assessment may also take into account assessments already completed, and the competencies demonstrated in these assessments. By looking at the context, you can consider the conditions under which evidence for assessment must be gathered.
• All activities are conducted adequately using the required:
• equipment or material requirements
• contingencies
• specifications
• physical conditions
• relationships with team members and supervisors
• relationships with clients/customers
• timeframes for completion.
• Assessment methods or tasks are suitable to the requirements of the units of competency and students are assessed on the tasks and activities according to the requirements of the training package.
• The language used is simple English
• The evidence required to make a decision of competency is clearly outlined
• The types of activities and tasks students need to perform are clearly outlined
• The level of performance required for each assessment activity is clearly outlined
• Adequate exposure to workplace conditions, including appropriate simulated environments, is provided
• Sufficient knowledge-based assessment tasks and activities such as written questions and case studies etc.
• Sufficient practical based assessment tasks and activities such as projects, role plays, workplace tasks and observations etc.
• Assessment resources are error-free and free from any grammar, copyright or plagiarism issues

9. Easy to contextualise to your learner cohorts 

Contextualising assessments and learner materials is one of the most thought-provoking tasks that RTOs face. Many RTOs are deemed non-compliant due to their contextualisation methods.

Regardless of whether you have developed the assessment resources in-house or you have purchased them as off-the-shelf resources, you must customise and contextualise each training product.

The customisation and contextualisation should occur in terms of

• training context,
• learner characteristics,
• delivery modes,
• cultural context,
• technology requirements,
• AQF level,
• intent if the unit of competency is not addressed appropriately,
• formatting,
• grammar,
• Your RTO’s templates and style guides

10. For more information, please read the following articles

https://caqa.com.au/2021/10/04/contextualisation-why-you-must-contextualise-the-training-and-assessment-resources/

https://caqa.com.au/2018/06/08/contextualising-assessment-resources-part-2/

How to make a reasonable adjustment in summative assessments

https://caqa.com.au/2021/07/14/how-to-make-reasonable-adjustment-in-summative-assessments/

Easy to modify to different delivery modes

Assessment resources should be adaptable to a variety of delivery modes, including online, offline, distance learning, workplace, and blended learning.

Clear guidance to the assessor

Assessment resources should provide clear instructions to assessors in terms of what they should look for.

• what was expected from the learners
• what they (trainers and assessors) should observe
• have clear assessment (evidence-gathering) methods based on training package requirements
• have clear assessment (evidence-gathering) tools
• how they (trainers and assessors) should assess
• have clear benchmarks and standards against which a student’s work is assessed
• have clear evidence requirements to assess sufficiency and competency
• have clear guidelines related to when assessments should occur and how they should occur
• who should collect the evidence and when?
• When determining who can collect evidence, the guidelines for training package assessment may be of use to you. It is critical that the instrument and instructions for your assessment tools clearly state what is expected of the students, the trainer/assessor, workplace supervisor or a third-party evidence gatherer. It is also critical that the instrument and instructions for your assessment tools provide a clear structure for the evidence gatherers to follow.
• where they (trainers and assessors) should record
• Focus on the requirements of the training package - are there any specific conditions, requirements and guidelines?
• If workplace assessment is not possible or suitable, your alternative is to choose settings and procedures that allow students to demonstrate their competence to the level of performance indicated.
• In a simulation, students should complete or deal with a task, activity, or problem in an off-the-job situation that is designed to mirror the workplace environment.
• how they (trainers and assessors) should record
• What are the RTO requirements and documentation?
• who they (trainers and assessors) should report
• Where do the assessors and trainers report the outcome of the assessment?
• What is the procedure?
• How students are informed about the assessment outcome?

Clear instructions to the learner

All assessment materials should provide very clear instructions to the learners in terms of:

• What
• When
• Where
• How
• Why
• Reasonable adjustments required

Observation checklists and benchmarking

All assessment resources should have comprehensive observation checklists and benchmarking to ensure

• Learners know what is expected of them
• Assessors know the scope of assessment in terms of what they should observe and assess, resources required and what should be considered when assessing
• Clear guidelines and information related to how to use the observation checklists

All equipment, resources and facilities are available to conduct the assessment

Assessment conditions should be followed strictly and the training organisation should ensure that all equipment, resources and facilities are available to conduct the assessment.

Language, Literacy and numeracy requirements of the unit

The assessment tool must reflect the language, literacy and numeracy requirements related to the work task and work activities required to be assessed. Your focus points should include:

• Reading
• Writing
• Numeracy
• Oral communication
• Learning

Ensure students are ready for the summative assessment

You must ensure that all students are ready for the summative assessment before you assess them.

For more information, please refer to

Formative vs Summative Assessment: A Comparison https://caqa.com.au/2022/01/31/formative-vs-summative-assessment-a-comparison/

Conduct pre and post-validation checks

You must pre and post validate all assessment resources before and after you use them for your learners.

For more information, please refer to:

Different phases of assessment and learner validation processes (Part 1), Click here.

Different phases of assessment and learner validation processes (Part 2), Click here.

Different phases of assessment and learner validation processes (Part 3 of 4), Click here.

Different phases of assessment and learner validation processes (Part 4 of 4), Click here.

fact sheet, Click here.

Having a license of purchase

Training organisations should double-check that they have an authorised copy of the training and assessment resources from the publisher of the resource before using them for training purposes.

In a number of audits, the regulatory body has requested proof of purchase because a number of stakeholders are aware that there are some offenders in the sector who do not purchase the actual copies of the resources, resell when they do not have authority, or obtain materials in other illegal ways.

For more information, please click here.

How to protect the copyright of your training and assessment materials Margaret Ryan (lawyer and trade marks attorney), for more information click here.

References:

Guide - developing assessment tools https://www.asqa.gov.au/resources/guides/guide-developing-assessment-tools