Starting the design process

Programme and course design (also known as curriculum design) is undertaken for a number of reasons.

Sometimes external criteria (such as accreditation, government or university criteria) change, and necessitate a change in what students learn and are able to evidence.

At other times, redesign reflects changes in pedagogical or conceptual approaches in the discipline itself.

Sometimes you may want to develop a completely new programme. Whatever the reason for engaging with curriculum (re)design, it is helpful to think of it as a systematic process. In this process you should:

1. Develop your programme- or major-specific graduate attributes. These should be aligned with Victoria University of Wellington’s Graduate Profile and any degree, qualification or major attributes defined by your Faculty.
2. Review and map the structure of the courses comprising the programme, including those offered by staff outside of the programme team to ensure programme attributes are mapped to courses and progression pathways identified.
3. Review and develop course learning objectives, and align these with assessment activities (formative and summative) and teaching/learning activities.
4. Develop a process for gathering evidence of the quality and effectiveness of the programme for supporting student achievement of the programme graduate attributes.

The Centre for Academic Development works with Faculties and Schools to assist academic staff in the (re)development of programmes.

The objective is to ensure that programmes are designed to effectively and efficiently provide pathways for student learning and attainment of the goals of their chosen major.

Redesign resources

• Developing programme-specific graduate attributes
• Mapping programme and course structures
• Reviewing and developing:
  • Course Learning Objectives
  • Assessment
  • Learning and teaching activities
• Evaluating your curriculum

Helpful resources

• Victoria University of Wellington’s Graduate Profile
• Programme and Course Design Handbook

1.3 Principles

Assessment at Te Herenga Waka—Victoria University of Wellington is based on six key principles:

1.3.1 Validity

Assessment should be fit for purpose. Assessment tasks should therefore be appropriate for the level, content and learning objectives of the course and the graduate attributes of the programme and university. A valid task will be one that measures what it purports to assess.

1.3.2 Reliability

Assessment should provide an accurate and consistent measure of student performance. This involves both consistency in marking and the authenticity of student work.

1.3.3 Fairness and inclusivity

Assessment tasks should provide every student with an equitable opportunity to demonstrate their learning. Tasks should not discriminate against students on the basis of gender, race, ethnicity, religion, disability or political affiliation.

1.3.4 Contribution to learning

Assessment should be recognised as a learning activity. Assessment tasks should contribute to the development of skills and knowledge that can be applied within the course as well as in other contexts.

1.3.5 Manageability

Assessment tasks should be reasonable and practicable in terms of time and resources for both students and staff.

1.3.6 Transparency

The intention and practice of assessment should be clearly described to students and to other staff teaching in a course so that its benefits, purposes and procedures are understood by all parties, in the spirit of a teaching and learning partnership.

Designing assessments

The information given here is generalised to cover most types of out-of-class assignments.

a) Select assessment tasks on the basis of their relationship to the learning objectives of the course. Be clear and specific in the descriptions of the task as to how it will support student achievement of the learning objectives and its relationship to other elements of the course.

b) Reinforce the value of time on task: make sure that the weighting of assessments tasks (grade points and workload) are consistent with the importance and scope of their relationship to the course learning objectives.

c) Ensure that the wording of questions, including all instructions, is clear and unambiguous.

d) To avoid confusion, instructions and questions should be separate from background or context information.

e) Begin each instruction on a new line so that students do not overlook or miss an important part of the task.

f) Emphasise key words (eg. use bold or capital letters). Any use of a negative word should be emphasised because it may be missed. Never use double-negatives in the wording of your questions: these are confusing.

g) Identify the scope of the task. Where appropriate:

• Specify the role to be adopted by students.
• Identify the audience (who the students are writing for).
• Itemise what students are to do.
• Specify the approach expected.
• Identify the medium for the students’ answers (eg. a feature article in weekly newspaper).

h) Include key assessment criteria such as the word length, key dates, and the weight that the assessment contributes to the final grade.

i) Specify the ground rules to students (penalties, word limits, late work). These should be included in the appropriate place in the course outline.

j) Ensure that a colleague or trusted third party reviews any instructions or material associated with the assessment, including specific questions, to ensure that they are clear to other people.

k) Discuss the details of the assignment with students in class. This will help to ensure that they have understood what is expected of them.

l) If any adjustments are made to the question or criteria, all such notices should also be given as announcements on the course Nuku site.

m) Provide clear information to students on where they can go for assistance if they are struggling with the assessment task or would like independent feedback. Commonly this will involve the staff of the Student Learning Support Service.

Figure 1—Example of the layout for an assignment

Marking guides and rubrics

To help ensure that assessment is a reliable, valid and transparent measure of student learning, a marking guide or rubric should be prepared. This should be made available to students and discussed with them before they begin work on their assessment task. A marking guide is also particularly valuable for situations where the marking is shared between more than one person.

The marking guide should set out the principal points you are expecting along with mark allocation. It may be helpful to prepare an abbreviated model answer. For example, what are the main points students should cover, expectations regarding style and structure along with any other considerations such as originality. The notes you make will help you decide which form of marking guide will be most appropriate. The marking guide should help students and markers understand the qualities of the submitted work that will distinguish elements that are acceptable and elements that are excellent. It should also identify any criteria for unacceptable work.

The marking guide should set out the principal points you are expecting along with mark allocation. It may be helpful to prepare an abbreviated model answer. For example, what are the main points students should cover, expectations regarding style and structure along with any other considerations such as originality. The notes you make will help you decide which form of marking guide will be most appropriate. The marking guide should help students and markers understand the qualities of the submitted work that will distinguish elements that are acceptable and elements that are excellent. It should also identify any criteria for unacceptable work.

Make clear in what way the course learning objectives are being assessed. If several objectives are being assessed, make sure that the guide addresses them. Alternatively, if students are being assessed on how well they are integrating several objectives, make sure the qualities listed in the guide (eg. ‘coherence of arguments’) reflect the objectives you want to integrate.

Identify the qualities students are expected to demonstrate in their answer or performance. These qualities may cover knowledge, skills, personal attributes, values, and attitudes.

Qualities that an essay might test

• relevance to the question
• organisation of answer
• coherence of arguments
• use of primary sources
• originality of thought
• fluency in writing (expression, grammar, spelling)
• citation and referencing practices.

Qualities that a design project might test

(eg. the logo for a new organisation)

• integrity and alignment to design rationale
• clarity/simplicity of design
• functionality (meets its purpose)
• cost effectiveness (if used commercially)
• originality
• evidence of research in creating the design
• aesthetic quality, innovation and creativity
• presentation (the way in which the presentation effectively communicates the design).

Allocate marks in proportion to the relative importance of the objectives and skills being assessed. To achieve validity you need to reflect the importance of the material that has been taught. Where possible, ensure that weighting of marks aligns with the workload experienced by students.

Consider allocating a small proportion of the marks to a category that reflects a holistic judgment of the work. The basis for allocating these marks (novelty of ideas, overall quality) should be clearly explained.

Communicate assessment requirements to students

• You must give out the marking guide to students before they begin their work so that they can know what is expected of them and how they will be assessed.
• Make sure the students have the opportunity to understand the question and the criteria that will be used to assess them, including opportunities to discuss these with staff and other students.
• Provide students in advance of deadlines with examples of work that embodies specific qualities associated with excellent, average and unacceptable work in order to guide their own efforts.
• Students must be told fully of every factor affecting marks.

Provide students with a statement on the grading criteria used in your course. This should identify in general terms what a student is required to do to get an 'A', 'B', etc. for the course.

The marking criteria used for each assignment should complement the grading criteria for the whole course.

Make sure to draw students’ attention to the School/Departmental policy on penalties such as lateness and university policies on plagiarism and academic misconduct listed in the course outline.

Students submitting drafts for feedback

Where possible, students should have the opportunity to submit parts of their work for feedback prior to final submission. Submission of drafts for feedback can be either a requirement or an optional support, in either case the ability of students to obtain this feedback should be clearly communicated to all students, preferably in the course outline as well as the assessment documents.

Encourage students to self-evaluate their work

Encourage students to self-evaluate their performance using the marking guide or rubric before they hand in their work. Research (Riordan & Loacker, 2009) shows that the ability to self-evaluate the quality of one's own work is a characteristic of top performing professionals. Consider making self-assessment a requirement: that is, have students hand in the guide containing their own self-evaluation. The spaces for comments can be used by students to justify or explain their evaluation.

Marking processes

a) Individual assessment items may be marked using letter grades or numbers. The pass/fail approach should normally only be used for small assessment items.

b) All assessment tasks should be designed and marked against criteria linked to the learning objectives of the course and not marked simply to a standard curve of achievement.

c) Where possible, students should be given basic information on how each assessment item will be assessed before they need to begin working on it. For tests and examinations such information might only be an indication of the distribution of marks across questions or between different sections. For essay type items it would be appropriate to assign marks to aspects such as structure, clarity of language or referencing.

d) It is expected that examiners (see section.1.4.7) will mark at least 50% of each student’s work in a course.

e) Normally no more than 50% of each student’s work in a course may be marked by tutors and others covered by the Tutors’ Collective Agreement. However, a higher proportion may be marked by tutors, where the work involves multiple choice or short answer questions with just a few possible answers.

f) It is essential that schools take active measures to ensure new staff who are assessing students are given adequate training, guidance and monitoring (see section 1.4).

g) The use of a marking guide is mandatory where the marking is shared between markers or undertaken by tutors.

h) Course coordinators must ensure that all those engaged in marking share the same assumptions about what constitutes an A, B, or C and agree on the interpretation of the marking guide and the weighting of marks. This can be achieved by processes such as:

• calibrating against marking criteria prior to marking
• independently assessing three or four assignments and discussing the outcome
• calibrating a sample of the first items assessed against a sample of later items.

i) Schools must have processes in place to ensure appropriate moderation of results is undertaken before marks for an item of assessment are released to students.

Moderation

Each Faculty has developed their own Assessment and Moderation Requirements in alignment with University guidelines.

Moderation might include:

• checking a sample of scripts from each marker for variation in marking
• standards by a given marker within their allocated scripts and also between markers
• examining the overall grade distribution and reviewing work at grade.

Course grade ranges and indicative characteristics

Course grade ranges and indicative characteristics in the Assessment Handbook Appendix B:

Attendance and assessment based on participation

a) Marks should be limited to specific forms of academic performance. This means that marks must not be allocated or deducted for simple attendance or nonattendance at learning/teaching events.

Tutorial and seminar participation

Forums such as tutorials or seminars provide students with the opportunity to develop the communication skills that are a vital part of everyday working life. Contribution to class discussion can help demonstrate course learning objectives, but assessment of that may be problematic.

Participation can:

• encourage students to prepare for class and to do the background reading
• encourage students to think and reflect on issues and problems
• foster the development of communication and presentation skills, including speaking and listening skills
• encourage social interaction and the sharing of ideas
• develop group and team skills (where the assessment focuses on group work).

b) Up to 10% of the assessment in a course may be based on participation providing:

• i) performance is assessed on clearly defined tasks and not on vague impressions of the quantity or quality of a student's contribution to class discussion;
• ii) criteria for assessing the in-class performance of students are clearly specified in a form that students can translate into action or behaviour;
• iii) students are provided with prior opportunities to learn the skills that are being assessed; and
• iv) records are kept for each class in which participation is assessed.

c) The Academic Committee may approve a larger component of the course assessment to be based on participation.

d) Tutors responsible for managing tutorial/seminar groups in which participation is to be assessed must have completed the appropriate CAD training on assessment and feedback.

e) When using participation as an assessment tool, consideration should be given to ensuring the assessment is fair to all groups and not distorted by gender or cultural differences, personality, class size, group dynamics or other factors that may affect students’ willingness and ability to participate.

Peer assessment

Peer assessment (assessment by one or more students of the work of another student enrolled in the same course) is useful to assist student learning and to develop a broader range of learning partnerships.

With the approval of Academic Committee, peer assessment may contribute up to a maximum of 10% towards the formal assessment determining a course grade.

Training and guidelines for staff in the design and conduct of assessment tasks that involve peer marking are important and course coordinators introducing peer assessment would be expected to work closely with the Centre for Academic Development (CAD).

In the first instance contact the relevant faculty liaison person from the CAD team.

Self-assessment may not contribute to course marks (section 2.2.3):

Self-assessment

A self-assessed grade must not contribute directly towards the formal assessment determining a course grade.

Self-evaluation

Self-evaluation is encouraged as part of reflective practice and where possible should be an integral part of all courses. A necessary component of self-evaluation requires students to reflect on and analyse their learning from the assessment task, and does not involve them assigning a grade (ie. it is not self-marking).

Although a student-initiated grade must not contribute directly towards the formal assessment determining a course grade, participation in the self-evaluation activity could be given a component of the course grade assigned by the lecturer.

Methods of feedback to students must be carefully planned to assist in improving the quality of learning. Feedback is often described as summative, providing information on the standard or quality of student work, or formative, providing information on how the student can improve.

Effective feedback

Effective feedback is timely; useful in that it provides information on strengths and weaknesses; provides clear information on how to improve; and encourages rather than discourages the student. Feedback that exposes misconceptions in the students’ models of understanding such as Concept Tests (Mazur & Somers, 1999) or activities that explore Troublesome Knowledge (Meyer & Land, 2003) can be very effective.

Seven principles of good feedback practice

The UK Higher Education Academy has identified seven principles of good feedback practice (Juwah, et. al., 2004) stating that it:

1. Facilitates the development of self-assessment (reflection) in learning.
2. Encourages teacher and peer dialogue around learning.
3. Helps clarify what good performance is (goals, criteria, standards expected).
4. Provides opportunities to close the gap between current and desired performance.
5. Delivers high quality information to students about their learning.
6. Encourages positive motivational beliefs and self-esteem.
7. Provides information to teachers that can be used to help shape the teaching.

Feedback strategies

A helpful strategy for designing or improving any assessment activity is to consider how the student might use feedback in a productive way to demonstrate that they have learnt and improved, within the current course context. Where possible, students should be given the opportunity to gain feedback in multiple ways while undertaking an assessment, either through the use of staged tasks or review of draft work.

Effective feedback strategies should consider the following guidelines:

• The feedback cycle begins when the students first see the assessment programme. If the information they receive about this is clear and accompanied by details of marking criteria, students are more likely to produce good results. Students and teachers have different perceptions of feedback (Carless, 2006) and it is most helpful if a dialogue between teacher and student can be developed around assessment.
• In order to give very quick feedback on a written assessment, consider giving whole class feedback when you have read a small number of the assessments. Or, you could provide a model answer on Nuku once all the work has been handed in.
• If more than one person is providing feedback ensure that your comments are consistent in their focus, detail and extent. A useful strategy, particularly when using multiple tutors, is to compile a set of standard feedback statements that all markers draw upon when addressing common issues. This can often be compiled easily after an initial sample of work is marked.
• Summarise your comments and indicate the fact that it is a summary. Try to be as clear as possible in your choice of language and use specific examples instead of generic ones where possible.
• Make sure your comments are legible. Ticks, ? and underlining are not very helpful to the student. They are your shorthand, but explain them to the student. Avoid terms or jargon, which students may not understand unless they are necessarily relevant to the material being learnt.
• Balance the positive with the negative. Both negative and positive points should be constructive and provide guidance for further improvements.
• Feedback is more effective if it is provided in multiple formats. As well as written feedback, consider providing feedback in lectures, through tutorials, online and individually. This can include podcasts or video provided through the University’s VStream system.
• Giving feedback is not the exclusive domain of the instructor. Encourage students to evaluate themselves and to seek feedback from other sources, including through discussions with other students. Peer feedback (see section 6) can be very helpful and constructive for students and may or may not contribute to, or influence, the final mark. Consider allowing time for this in the tutorial session when assignments are handed back. Students will also obtain feedback through the experience of collaborative tasks undertaken in class, including in lectures.
• When working with smaller classes, postgraduate students or in response to a draft piece of work ask the students what kind of feedback they want. For example, do they want you to provide specific comment on structure, the introduction, etc.?
• Make sure that there is a clear relationship between the grade and the feedback provided. Use of a rubric or guide (see section 2) can help provide context. Ensure your marks and comments are consistent with the criteria and reflect the appropriate weighting (i.e., 'not a satisfactory piece of work because<: grade D‛).
• Distinguish between different skills (e.g., student may have lots of good ideas but may be poor at spelling).
• Give suggestions for possible follow-up to the work where appropriate (e.g., further reading or extending ideas).
• Use technology to make giving feedback easier. You could use Word’s 'Track Changes‛ feature to comment on assignments which have been submitted electronically. Or you could post general class feedback on the course Nuku site so everyone can benefit.
• Consider using a feedback cover sheet or handing back a summary of the most common problems in the class as part of your feedback.
• Keep a copy of your comments for each student. When marking the next assignment check to see that students have improved/taken account of earlier comments.
• Getting feedback on feedback can help the marker improve their practice, or the lecturer improve the lecture.
• Allow time when you hand back the assignments for students to read your comments. Ask them to identify one point that they will work on in the next assignment. Allocating time signals to students that reflecting on feedback is a valued activity.

Feedback pro forma

The following feedback pro forma provides an example of how students can be encouraged to systematically consider their response to feedback. To increase the usefulness of this approach ask students to make two copies of the completed pro forma. One copy can then be handed to the tutor/lecturer for later reference.