Sunday, December 12, 2010

Final Report is Finally Complete

Dear All
Whew what a marathon.  We are done. 
Please follow the link below to view my report in its PDF format.
http://electronic-learning.yolasite.com/final-report.php
Your comments are much appreciated.

Have a lovely Chrismass and holiday.
It has been lovely working with you.

Louise.

Tuesday, November 30, 2010

Almost at the conclusion

Hi all
It is a while since I posted anything on my blog but Dana and I have been working away at our final report.   He will not participate tonight as he is unwell but we hope to post our work next week.   How is everybody else doing.  Both Dana and I have been heavily tied up with students sitting their examinations either preparing them for them or marking the final results which all have deadlines.  Hence the lack of particpatioin on the blog.  I have looked up some material related to evaluation from overseas.  One article by Fetaji & Fetaji (2009) discusses a multidimensional evaluation of elearning using a qualitative research design and comparative analysis of factors that can effect student learning utilising two LMS - ANGEL and Moodle - Doies anyone have any experience of the ANGEL LMS?   This is an interesting report identifying factors that most of us would know about such as learner styles, culture, skill level , teacher effectiveness, intelligence variation, communication styles etc.  This report was located as an Electronic Journal of e-learning, 7(1) 1-84) but I could not find a tital except it focused on evaluation of elearning.
HOpe everybody is making good progress.  Louise   

Sunday, September 12, 2010

My evaluation plan

Hi everybody

I am posting my evaluation plan for assignment two.  I have done a literature review but cannot access it so will post it later.  I am at present in London at a conference.   I would be pleased to get comments
Dana and from all the group. Hope everybody is going Ok with theirs. 

Dana I could not get our pictures to copy!!


Introduction
In 2009, the writer enrolled into the Graduate Certificate of e-learning and in the Design paper of this qualification, an on-line Conception to birth component was developed and posted on the MIT Learning Management System for student learning. The first time this component was run was in Semester one, 2010 and it became evident that there should be some modification if the students were to use this component and learn from it. In semester two and a new group of students, a modification has been carried out.

The purpose of this evaluation plan is to evaluate the modified e-learning component of the Lifespan development course “Conception to Birth” to see if it as an effective learning tool.

Background

The design of the component is directed by the Blackboard Learning Management System (LMS) and uses the format the students have been introduced to through EMIT. Students are expected to access the LMS through their User Login and password that they receive on enrolment. They are also taught as part of their orientation how to navigate around the system. They are informed that all announcements, course information, course content, and assessment information will be accessed via the LMS. Students become familiar with the LMS set up and expect to see information presented in the design of the system. They can access the system asynchronously any time day or night.

From Conception to Birth is designed around a story of a family expecting a new baby. The family consists of father, mother, an elder son and baby daughter and an expected new baby entering the family (appendix i).

Students are directed to a course text from which the story of the family acts as a background for numerous questions that trace the conception to the zygote, embryo and fetal phase of the pregnancy. They are given online resources that illustrate the development process, the timeline involved and critical phases with questions to assist the students to find out the information they need to for their learning.

Additionally, a number of technological tools are used to assist them in their learning process. They have a number of ‘matching’ exercises to learn key terms, Quizlet is used for learning new terminology, a crossword is used to link concepts to definitions and a summative multi-choice test to see how they were learning. The summative multi-choice test was worth 10% of their total mark for the paper and comprised 20 multi-choice questions. From the total enrolment only four students failed to reach a pass mark and the highest mark was 19/20 and the lowest 9/20.

A discussion board was also set up for students to discuss the component as they learnt the material. In the original exposure there were 148 students enrolled in the course. To manage the large number of students, they were placed in 16 discussion groups. Instructions were given verbally, demonstrated in class and placed within the conception to birth component with an appropriate access button. Each group was asked to choose a leader who would take the week’s discussion from the group and place the key response themes on a class wiki. The writer would then check on each of the wiki’s to gain an overall impression of how students were learning and more importantly, how students were accessing and using the systems that were available to them to assist them with their learning. The discussion board did not contribute to the overall course mark for students. The discussion activity was broken up into four weeks with questions to drive the discussion each week. The writer kept a check on the discussion activity of the students.

A summative evaluation was undertaken at the conclusion of the component. Students were asked to evaluate their experience with the component and sadly only four students completed the evaluation. These four were positive.

From the writer’s observation, the discussion area of the learning component was not utilised well with very few students using it and the leadership and class wiki was only taken up by one leader. It was decided on observations made, that the component would be modified for the new enrollees for semester two.

Modifications made have included more instructions given frequently to students through the Emit LMS. Checking in class to see how the students are managing. The leadership for group activity has been scrapped along with the wiki but the discussion groups have continued with the questions being announced and published weekly to guide the students. The writer checks on all groups twice per week to see how the activity is being managed. To date I have noticed that this monitoring is making a total difference to the uptake. The students are enthusiastic and I monitor how frequently individual students access the discussion. The very new students are beginning to ask their own questions apart from the formulated ones which are very encouraging. The summative multi-choice has been changed to a formative test which is for their own monitoring of their learning but the students have been informed that questions related to the component will appear in their summative test to contribute to 20% of their total mark for the paper. Additionally the students have been informed that the group who has the most participation on the discussion board will get chocolate fish. This really has got the students going.

Aim:

The aim of the evaluation is to see if the modified e-learning component of the Lifespan development conception to birth is an effective learning tool for new students of nursing.

The purpose of the evaluation was threefold:

(a) to determine whether e-learning can satisfactorily replace the traditional lecture method of transmitting theoretical information to new students of nursing.

(b) To determine whether the conception to birth component was designed in an effective manner for student learning?

(c) To assess the effectiveness of the component through both formative and summative evaluation to demonstrate student learning.

Methodology:

Evaluation big questions:

Matched to purpose

1. How effective is the on-line segment of the lifespan curriculum component “conception to birth” as a tool for student learning?

2. How effective was the formative learning tools to assist their learning?

3. What did their formative test results demonstrate in relation to the material to be learnt?

4. What did their summative test results demonstrate in relation to the material to be learnt?

Related questions:

1. Was the timing correct for the students to respond to the questions asked:

2. Did they find the reading material interesting?

3. Did they find the questions easy to answer from their text book reading?

4. How easy was it to access the discussion board?

5. Did they find the discussion board helpful to their learning?

6. Were the questions asked relevant to their key learning goals?

7. How did fellow students assist their learning through their participation in the discussion board?

Data collection methods:

Design of evaluation

It is proposed to use a range of data collection methods to evaluate the effectiveness of the on-line conception to birth component of the lifespan curriculum content. These are as follows:

1. An evaluation form has been developed as part of the component and students are asked to complete the evaluation form (appendix 1)

2. Lecturer assessing the discussion board regularly over a two week period to assess participation and to assess the quality of the responses to the questions asked

3. Lecturer assessing the formative test participation and results of the students who participated

4. Lecturer assessing the summative questions as part of a formal test of the lifespan component of the curriculum to include conception to birth (10 questions)

5. Students completing an evaluation of the conception to birth component of the lifespan content

Audiences

1. Students undertaking 722.520 Foundations of Nursing (Lifespan Conception to birth) component are the primary audience

2. Myself as the primary lecturer

3. Secondary sources other lecturers involved in the teaching of the Lifespan component

4. Bachelor of Nursing Teams One and Two Co-ordinator

5. Programme Leader of Bachelor of Nursing programme

Logistics and Timeline

Implementation

Instrumentation

1. Evaluation form

2. Multichoice questions – formative

3. Summative


Sample

1. 112 students

2. 6 lecturers

Timeline = one semester 20 weeks.

1. Discussion board open for four weeks – the month of August, 2010

2. Analysis of responses on discussion board for 12 groups of 10 students twice per week and responding to the students 8 hours per week = 40 hours

3. Monitoring hits and performance on formative test of participating students –August to September 3 - 4 hours per week

4. Summative test results = 8 hours

5. Evaluation of component per evaluation form yet to be determined – 4 hours

6. Approximately 56 hours estimated in the evaluation process

Budget – 56 hours at $45 per hour = $2,520

Decisions

Outcomes expected:

Positive outcomes would be:

1. Students participate fully in the discussion board

2. Students take advantage of the formative test for their own learning and their is at least a 50% participation rate as measured

3. Students demonstrate 100% achievement of the ten related questions to the conception to birth component of a forty question summative test worth 20% of their grade for the Foundation of Nursing paper

Negative Outcomes could be:

1. Poor participation (<30%) in the discussion board

2. Poor participation (.30%) in the 20 question formative test on the conception to birth component of the Life Span teaching

3. Poor pass rate (>30%) on the 10 question component of the lifespan 40 question summative test

From the total process – further modification to the component a conception to birth of the lifespan teaching according to the demonstrated performance of the students and the feedback on their evaluation of the component.

It would the same for the lecturers involved in the teaching and the summative test results.

Sunday, September 5, 2010

Working with Dana

I have been working with Dana on our assignment on Friday September 3.  We have had many useful discussions one of which was to clarify the distinction between an aim and a purpose.  This seems a grey area but I believe it is that an aim is a more broader approacvh to evaluation in relation to our topic and that the purpose is more specific.  That is the approach that we have taken. 

Unfortunately in the LSC interruptions occur with students wanting assistance and making appointments for that is the focus of the centre so we get quite a few interruptions.  Nevertheless we are making progress.
Dana's smiley face approach to a difficult topic will I am sure be a winner with students especially as they create their face and require to know the constituents of the cellular structure in order to do this.  Therefore there learning will be accelerated through both their research, finding a solution to the problem confronting them and drawing their smiley face.  This approach to learning is fun and should make it easier for them to remember the structure of a cell.

I am getting along with my assignment and hope to post this tomorrow.  I have done a literature search and have some interesting reading that Dana and I will share in discussion and select what is relevant for both of us. 
HOpe everyone else is getting along OK. 

Dana is a great coffee maker!!!
Louise

Wednesday, September 1, 2010

Working on my assignment

I am working with Dana in the LSC on our assignment. 
WE are considering the various dimensions of the work together and are applying it to our different projects.  We recognise that our discussion surrounds the process and we are working through this.  I have written quite a lengthy background and introduction to establish the context but will need to edit this.  It is helpful in the beginning to clarify my own thinking so by writing out my own thoughts it accomplishes this.  We are moving on this evening together through the process. 

Monday, August 9, 2010

Case Study

Case Study



'Evaluate this' exercise - Online Information Literacy modules


Some online modules have become available on the Internet as the result of a national project. They were developed to help students and teachers learn to manage online information more effectively, and are called the OIL (Online Information Literacy) modules. A variety of evaluation processes were carried out during the project.


Choose one module to evaluate using the following questions:


• How useful is the module for your work situation? Explain why it is or isn't, and how you would use it.


• Comment on the look and feel of the module and how 'fit for purpose', or not it is.


• What sort of changes would you make if customising the module.


• Other comments.


o Why is evaluation important and how do you define it?


o Which two elearning Guidelines suit your context?


o Tell us how the 'Evaluate this' exercise went from your perspective. For example, which module did you evaluate, how did you do this, and what did your group decide, and did you agree?

The module we have used is about Essay writing. We have chosen a number of significant screenshots to demonstrate the sequential nature of the essay writing process.


Our responses are found following the screenshots.



 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
You have completed the Understand the topic section of the module. You should now understand what the following words mean:


• Instruction words are the action words in the topic and tell you what to do.

• Content words are typically nouns and verbs that represent the key concepts of the essay.

• Relationship words give you a sense of the connection between different aspects of the essay.

• Remember: If you want to go over any of these words again, you can return to any of the sections at any time during the module, either by using the green arrow to the left of your screen or by clicking on the table of contents to the left of your screen.

• Some people learn by seeing (visual)

• Others learn by hearing (aural)

• others by processing text (reading/writing), and

• others learn by doing (kinesthetic)


Question One:

What positive impacts has the Treaty had on New Zealand? (from the literature)

Question Two:

What negative, or variable impacts, has the Treaty had on New Zealand? (from the literature)

Question Three:

What positive impacts has the Treaty had on New Zealand? (from my experience – family and personal)


Question Four:

What negative, or variable impacts, has the Treaty had on New Zealand? (from my experience – family and personal)

Question Five:

What are the implications for working with people? ( What aspects of the treaty need to be considered by people working with people?)

















 
 
 
 
 
 
 
 
 
 
 
 
 
Treaty of Waitangi


________________________________________

1. Impacts

A. Positive

1. Implications for working with people

B. Variable

1. Implications for working with people

C. Negative

1. Implications for working with people

2. From familial experience

3. From the literature

4. From personal experience



 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Summarising and paraphrasing can be useful tools in essay writing as they allow you to include other people's ideas without cluttering up your essay with too many quotations. Consider summarising or paraphrasing when an idea from one of your sources is important to your essay but the precise wording is not.


Do not rely too heavily on either summary or paraphrase. Remember, your considered ideas are what matter most, so allow yourself the space to develop those ideas.

Summarise:

It is a brief summing up of key ideas

A summary may be of a whole paragraph, or even a whole article

It may refer to more than one reading with similar ideas

You must acknowledge your source, both in-text and in the reference list

Paraphrase:

It is extracting the key ideas and putting them into your own words. This will usually be shorter than the original

The term tends to relate to small pieces of text

It can demonstrate your ability to understand concepts, but your essay should be much more than just linking together other people’s ideas

Minor changes in text, especially when not referenced, must not be used

You must acknowledge your source, both in-text and in the reference list

Section summary

You have completed the Gather information section of the module. You have tested your knowledge of different types of information, and learned about the following reading techniques:

• Survey to get a general idea of the material available

• Skim for a general overview of content and decide whether it is worthwhile reading deeply

• Deep read with full attention to understand and analyse

• Scan to quickly find specific information e.g. Treaty

You have also learned how to select and record information using one or more of the following techniques:

• Highlight - identifying important words and phrases using a coloured pen

• Annotate - making notes on photocopies of the information

• Make notes - recording information about what you have read

• Quote - using the exact words of the author to support your argument

• Summarise - a brief summing up of key ideas

• Paraphrase - extracting key ideas and putting them in your own words

In-text citation

When you use an actual quotation

When you summarise ideas from one or more authors

2. Reference list

You have been guided towards an essay structure through the planning examples provided. Here you can see how to assemble the information you have gathered into a logical essay structure.

We cannot provide an example essay for your topic, but have a look at the example essay below about the National Parks of Otago and Southland. It was written for The Cyclopaedia of Otago and Southland (Dunedin City Council, 1999) and was one of the thirteen essays that dealt with Recreation and the Environment. It is reproduced here with kind permission from the Dunedin City Council and Neville Peat.

Essays have a recognisable structure:

• introduction

• body text

• conclusion



Both parks are introduced Introduction: The parks are named, located and their international status is given.

Paragraph Two: The extent of Fiordland National Park (FNP) is given and access is described.

Paragraph Three: The relationship between landscape



Paragraph Four: Biological features of FNP are discussed.



Paragraph Five: Mention is made of FNP’s walking tracks.

Paragraph Six: The extent of Mount Aspiring National Park (MtANP) and its diversity is described.



Paragraph Seven: Landscape features of MtANP are discussed along with historical references.



Paragraph Eight: Access and biological features of MtANP are described.

Management of both parks is summarised. Conclusion: Management of both parks is outlined

You have completed the Construct the essay section of the module. You should now understand more about the following:

Essay structure

• introduction

• body text

• conclusion

Paragraph structure

• topic sentence

• explanation

• example

Grammar and style

• Active and passive voice – prefer the active

• Verbs and nouns – prefer verbs

• Coherence – move from general to specific, old to new, simple to complex

• Emphasis – place your strongest point towards the end

• Empty subject – avoid beginning sentences with words such as ‘it’ and ‘this’

• Language use – consider academic tone, audience and use of inclusive language

• Performance errors – proofread carefully

• How useful is the module for your work situation? Explain why it is or isn't, and how you would use it.

The module is very useful for essay writing for beginning students in the Bachelor of Nursing programme. Although some students are school leavers, many mature students have not written a formal essay before or not for many years. Within this group of students are a wide range of abilities and learning styles which the OIL module addresses effectively. The helpful guide takes the writer through all steps of the process from the instruction words that direct the writer to what is required of them to the final summary and conclusion of the process.

Each activity required in the process is identified for example, instruction, content and relationship words directing the writer as to how to identify the ‘how what and why’ of writing the essay.

As we teach and nurture students in essay writing at the beginning of the Bachelor of Nursing programme, the OIL module is extremely helpful and completely user friendly. The module reinforces the steps in the essay writing process and gives quite clear descriptions and prompts when the responder has identified the correct response to questions e.g. “Identify the instruction words in the topic. Click on them and if you are correct they will highlight”. The word then changes colour to identify a correct answer. There is immediate re-enforcement of correct learning.

• Comment on the look and feel of the module and how 'fit for purpose', or not it is.

The module is entirely user friendly – colourful, well signposted, each signpost has navigational

Properties to expound further the instruction that it describes. It fits the purpose extremely well of introducing the essay writing process to a new writer clearly.

• What sort of changes would you make if customising the module?

The module comprises quite extensive reading and students notoriously don’t want to read and review lengthy instructions and materials. A suggestion would be to shorten the amount of information included and reduce the number of pages required to be read.

Why is evaluation important and how do you define it?

Evaluation requires deciding on the purpose of the evaluation and can involve the following questions and responding appropriately:

 
 
 
 
 
 
Activities in the evaluation process require one to evaluate the design of the module which must include:


1. clear instructions as to what to do and how to navigate and use the course tools and materials e.g. a site map and online support

2. meaningful and engaging activities that will require interaction such as games

3. minimise the amount of text information that the student must work through to achieve their learning goals

The design in this instance requires the learner to work alone and therefore must be explicit to avoid poor understanding and an inability to achieve learning goal. The essay writing OIL module meets all of these requirements.

Which two E-learning Guidelines suit your context?
 
 
 
 
 
 
 
 
 
 
 
We have chosen the learner centred and teaching relationships.


We do believe that constructive learning relationships with learners is an important component of effective learner centred learning. For example, Dana remembers that teachers who constructed a positive learning relationship in a welcoming learning environment induced positive learning and a desire to learn more about the topic. Conversely, the opposite is applied when the teacher who was not learner friendly. Dana can simply formulate the following: you love the teacher, you love their topic. What Dana means by “love” is the following:

1. teachers accepting students for who they are

2. teachers making lessons fun

3. teachers having a sense of humour

4. teachers having a thorough grounding in their topic



Tell us how the “evaluate this” exercise went from your perspective?


We believe that the exercise (particularly from the taster) gave us insight into the evaluation process and into the depth of thought required involved.

It was interesting to identify the questions to be asked and to recognize the difference between “formative and summative” evaluation processes.
















Tuesday, August 3, 2010

Dana and I have worked through a Taster to use the concepts inherent in evaluating a elearning course.  Tomorrow we will undertake the case studies.

Questions:


1. What are the key issues?

2. How can the problem be solved? What strategies can be used?

3. How can the evaluation process assist in finding a solution? What type of evaluation is appropriate in this situation?

Career Development

A previously face to face course has been re-developed to cater for off-campus, distance students, working independently and completing assessments (e.g., preparation of a Curriculum Vitae (CV)). One of concerns noted during development of the online content was that the students might scroll too quickly through one of the important resources, the virtual business tour, so they can get their CV done quickly. This means they might not engage fully in the real learning aspects of the programme, and end up with an incomplete or poor quality CV. The educational developer is worried about changing a programme which used to rely on a face to face approach with students being stepped through the programme by a Tutor in class. There will be tutors facilitating the online course, but they are new to online teaching. The developer is not sure how the students will engage with the tutors, each other and with the online materials. The tutors are worried that they won’t know when the students are struggling. The developer is not sure how to evaluate the changes to the design of the materials and how to find out the level of support required for the students.

Key Issues:

1. A concern is that students might scroll too quickly

2. Redeveloped course has not been effectively designed to take in an e-learning context

3. The course design may not have sufficiently engaging activities to enthuse the learners to navigate appropriately through the necessary steps for learning to take place

4. New tutors may not be competently familiar with on-line teaching tools and skills and therefore are unable to assist the student meaningfully

5. The tutors may not be able to facilitate an online community

6. The most important issue is the lack of knowledge and skill in how to evaluate an on-line course, its design of materials and how to investigate the level of support required for the students

Solutions to the problems:

1. The new design must include:

(a) clear instructions as to what to do and how to navigate and use the course tools and materials e.g. a site map and online support

(b) meaningful and engaging activities that will require interaction such and games

(c) minimise the amount of text information that the student must work through to achieve their learning goals

2. Engage highly skilled tutors in e-learning to be involved in the delivery and support of the redeveloped course

3. Regular interactions and feedback by the highly skilled tutors during the delivery of the course

4. The availability of more than one tutor so that the student can have access to help quickly

5. The developer requires to do a course in evaluation of e-learning and ask specific questions such as:


How can the evaluation process assist in finding a solution? What type of evaluation is appropriate in this situation?


The developer needs to know more about:








I believe that important forms of evaluation should include both formative and summative. In the formative category one needs to explore how useable the site is to the students and to the tutor. As Well it is important to find out the most relevant information for the students and what that should include. This will cut down on verbosity and assist in both the useability and the relevance of the site. The developer also needs to know how easy the site is for students and tutors to access for otherwise both groups will not use it.


Summative evaluation will be used to establish effectiveness once the redevelopment has been concluded. The effectiveness of the site will establish that the intention of the developers, that is the learning outcomes have been met. Summative evaluation will also address how readily the site is maintained and is a measure of the efficiency and the effectiveness of the course in total. That is

If students can access it and navigate it successfully, find it user friendly and meeting their learning goals the developer knows that the site is well positioned.

It is important also for the students and the tutors to develop skills and a toolkit for using e-learning sites. The impact of the site, the learning incurred, the tools and skills developed will all be useful to transfer to new elearning situations.











Monday, August 2, 2010

Session Two

Dear group members
I attended the elluminate session last week with Bronwyn and met some of you on line.   Justs getting into the course at this stage and finding my way.  I found getting on to the elluminate site quite challenging but eventually got there.  I am working with Dana as my partner.  I am not so hot and still are a little 'fumbly'. We share the passion for learning.    

We are just going to get going on the 'taster' to introduce ourselves into the skills that we need to participate in the course.