A theoretical model that I have learnt about and used during my journey on the evaluation paper

Hi everybody

I am sure, like me, you have learned a lot of new ideas.  I did not know that so much had been written about the term'evaluation' and now recognise that there are many interpretations of this term.
Using Reeves and Hedberg's (2003) six facets of instructional product design I learnt that all of these require different questions and approaches.  The six facets, needs analysis, formative evaluation, effectiveness evaluation, the overall impact of the learning package,  maintenance and then finally a review of the total instructional process.   The most confusing terms for me were formative and summative evaluation as these terms I am very familiar with within a particular context.   The 'effectiveness evaluation' which was the focus of my study and report required me to look at the total learning package.  This I found was extremely comprehensive and complex.  Reeves and Hedberg claim that evaluating on-line instructional designs is a complex and an inexact science and I would totally agree with them.  Perhaps what was most significant for me was to re-orient myself to the different approaches to the word 'evaluation' and recognise that there are different orientation e.g. the orientation required for my 'effectiveness' evaluation was the methodological approach. 

During the theoretical exploration I looked anew at a research term 'triangulation' that I was familiar with from another context but began to understand it differently in relation to the 'effectiveness' evaluation.   I read up on and explored the electic mixed methods pragmatic approach to data collection to understand the origins of this approach and recognise that it has evolved from principally the logico-positivistic paradigm in the first instance, then the interpretive paradigm and finaly has a pragmatic dimension to it.  Reeves and Hedberg (2003) consider that using the electic mixed methods pragmatic approach to collect data for analysis on the topic of issue provides a practical orientation to practical problems that instructional designers are confronted with and provides useful information in order to move forward.  

My exploration into the electic mixed methods approach took me to Bronwyn's webpage (2003, (c) Otago Polytechnic  "Experimental and Multiple Methods Evaluation Models' and my reading of this literature extended further.  The term 'triangulation' became clearer and the techniques e.g. survey, interview, discussion board entries, as well as quantitative scales that show definitive outcomes.    The upside to this method is the breadth and depth one can achieve by examining your topic from a range of different perspectives.  For my 'effectiveness' evaluation I used a student survey, lecturer peers and a written comment to address the questions that I had raised to gain insight into the 'effectiveness' of my on-line 'Conception to irth' of the lifespan content that new nurses need to learn.  But of course the downside to this method is that the results achieved depends on a large enough sample to gather the data, the adequacy of the techniques to establish the depth and breadth of information required and then much of it is probabalistic rather than predictable.  However,  the triangulation approach does give comprehensive data from which to make decisions to points ways to improving teaching/learning processes. 

So as far as I am concerned, I learnt a lot but there is still more to learn no doubt.  

I will be interested to hear others' points of view.

Congratulations on concluding the evaluation paper.  I am sure you too have had an interesting journey.

All the best for your future studies.

Louise