• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!


Focus of evaluation

Page history last edited by Wallace Man-Yu 12 years, 5 months ago
  • Look at the table and the questions Brown raises (adapted here from Brown 1995, pages 234 to 239) and fill in the table in your own notes in the boxes instead of the questions  to see how this works for your course (suggestion: use a different colour of font to identify who has added what)
  • consider whether this would be a useful model for you to use for evaluating an online course. Would you want to modify this in any way? Your suggestions could include specific points about online courses.
  how effective, and to what degree? efficient in terms of time and reliability? attitudes of teachers, students and administrators?

 Which of the needs that were originally identified turned out to be accurate (now that the programme has more experience with students their relationship to the programme) in terms of what has been learned in testing, developing materials, teaching, and evaluation?

The students' needs originally identified had to do with the need my students had to establish real interactions with native speakers in order to acquire the target language.   

I predicted that the students needed a more efficient way of studying - in terms of time. Online learning negated the need to travel to and from a school/institute.  

prior knowledge

 the skill for applying a concept always has some "prerequisite" skills, skills which must be mastered before it is possible to learn any given  skill

 Which of the original student needs turned out to be most efficiently learned? Which were superfluous?

Some linguistic needs can be identified in the course easily.( e.g. accuracy)

However, there are some students who tend to seek standard answers to questions. Wiki or collaborated learning could be big challenge for them.

 What are the students', teachers', and administrators' attitudes and feelings about the situational and language needs of students? Before programme? After?

The attitudes of the students were very symathetic to a more 'traditional' methodology but now I think they appreciate task-based collaborative techniques.

A good proportion of students appear to be passive in online interaction. The socio-cultural and gender patterns of communication could be considered in needs analysis.


 Which of the original objectives reflect real student needs in view of the changing perceptions of those needs and all of the other information gathered in testing, materials development, teaching, and evaluation?

A major objective was to develop a sense of community and to promote  the advantages of task-based collaborative learning.

 Which objectives turned out to be needed by the students and which did they already know?

I think they 'knew' what community spirit was and what collaboration was but until they actually participate in these practices, they could not fully appreciate them.

 What are the students', teachers', and administrators' attitudes and feelings about the usefulness of the objectives as originally formulated? Before programme? After?

Skeptical --> Energised.


 To what degree are the students achieving the objectives of the course? Were the norm-referenced and criterion-referenced tests valid?

In order to evaluate if students were achieving the objectives of the course, I will not do any test but I have created a poll where students can do a self-assessment of what they are achieving.


I agree with Sergi. I don't give tests.Students should self-assess their own progress.


 Were the norm-referenced and criterion-referenced tests efficient and reliable?

I think these kind of tests may lack reliability and specially without a software like turn it in.

I think an online test within a limited time like those of ICDL would be more reliable and efficient

 What are the students', teachers', and administrators' attitudes and feelings about the usefulness of the tests as originally developed? Before? After?

If students want a criterion-based test, they can do IELTS or TOEFL or First Certificate etc. However, I think all parties in my institute are aware that a process based syllabus needn't have tests. Ultimately, the students will vote with their feet, so as long as the students are content with their own progress and the methodology, we can continue as their teachers. (The process of evaluation, obviously, is never ending).


 How effective are the materials (whether adopted, developed, or adapted) at meeting the needs of the students as expressed in the objectives?

I finally found a solution to the problem of integrating all these tools through the implementation of a social network (here)  where I could integrate the organization of groups, the activities, the blog ... 

 How can materials resources be reorganized for more efficient use by teachers and students?

This box has been left empty for two years and I think it explains something: Whether such resources can be used or how is it used depends on the system. SL learning is conditioned by how the administration of the SL learning is involved in the teaching process. This varies from system to system. 

 What are the students', teachers', and administrators' attitudes and feelings about the usefulness of the materials as originally adopted, developed, and/or adapted? Before? After?

Some people have got quite frustrated with the technology. It promises so much but, in reality, it can be energy-sapping - either because of the steep learning curve or that it simply doesn't work!


 To what degree is instruction effective?

For some of the students, the methodology might not seem to be so effective because it is new to them. Perhaps, if the method was followed over a period of time (e.g. one term), the skeptical students may be convinced of the methodology.



I agree ith Mark that as it is an online course, students might not get used to it. I understand this section as how effective would be the way we give instructions online: in my case, I have prepared clear instructions on what they have to do using a tool called splashcast where students can see what they have to do: an example is here  


Quick Feedback.

 If learners practice the wrong responses without being corrected, those wrong responses are likely to become difficult to change.  Thus, it is very important to correct wrong responses as soon as possible after they occur.  More over quick  feedback can serve a second purpose"motivation"


 What types of support are provided to help teachers? Are they efficient?


As an online course, an area unfamiliar to most teachers, a pre-course seminar covering suggested ways of teaching online in the context of this course will take place prior to commencement.


Technical support will be continually available.


Teachers working collaboratively would be another way of supporting each other.

 What are the students', teachers', and administrators' attitudes and feelings about the usefulness of the teaching as originally delivered? Before? After?

As a teacher in China, I face a number of constraint on this issue. So do my students and my administrators. It has always been about the tension between the need to pass the exam and the what the teacher thinks effective in SL learning. 








Comments (6)

Mark Hamilton said

at 2:27 pm on Mar 25, 2009

Is this the kind of thing?

Ruby said

at 5:55 pm on Mar 25, 2009

brilliant! yes!

Jez said

at 10:51 pm on Mar 31, 2009

Apologies for delay. I have been writing not 1, not 2, but 3 Research Methods assignments.

I think this is a useful framework for evaluation. Worthwhile using in tandem with course design.

Sergi Roura said

at 8:28 am on Apr 3, 2009

Hello everybody!
I put some suggestions at the table but I would like to receive some critical feedback as I am not sure on how to deal with it.

Ruby said

at 1:46 pm on Apr 9, 2009

This is building into a very helpful list of things we could include in an evaluation. I like the idea of a lot of self-assessments, although you may find that students will actually want to have a test! In terms of needs, it might be interesting to review the way you have all been working on the OLL course - I strongly suspect that at the beginning a lot of you were really wanting something very structured and tutor-centred, and that has perhaps changed? I am also aware that the change in study pattern has not been particularly successful for some, and I'm considering that next time I run the OLL course I'll have more tutor-led sessions (perhaps not once a week, though) for thos able to attend. Your comments would be helpful! Remember that I will also be evaluating this course :-)

Wallace Man-Yu said

at 1:06 pm on Jan 31, 2012

This looks easy. but it should be more collaborative. If only editors made more change to each other's change.
But dilemma here would be it becomes more linguistically collaborative but less communicative in terms of personal perspectives.

You don't have permission to comment on this page.