EDST5126 Week 3 – Learning organisations and academic leadership 

Well this is another huge topic and I have procrastinated blogging about it because there was a great deal of material to ponder and digest.

This week was quite content and concept dense with several new frameworks, concepts and matrices introduced which are useful tools that can be used to evaluate organisations, change processes and leaders within the context of exploration of issues and means of addressing them, in other words introducing strategies to deal with these issues, including in the higher education context.

In this blog, I will summarise the various frameworks explored, looking at overlaps and inter-relationships between them and apply them to the context of a post graduate speciality training program currently in a phase of flux and renewal and potential change. Having the change to ‘play with’ these models here will be invaluable when it comes to the second assessment, and more importantly as I grapple with and am involved with changes in higher education in the future.

Learning Organisations (LO)

I learning organisation is not, as one might expect, an organisation whose core business is learning, but it is in fact an organisation which learns, and is defined using principles pertinent to learning in the context of education, but applied to organisations.

Three quotes from Senge, Cambron-McCabe, Lucas, Smith, & Dutton (2012) resonated really strongly with me as I was grappling to understand the concept of the LO:

  1. “Learning is at once deeply personal and inherently social” (p. 4) (importance of focusing on the individual, the groups and the organisation, flattening of hierarchies)
  2. “If we are ready for them, living and learning are inseparable” (p. 4) (importance of continual growth, reflection and change and aspirations)
  3. “Schools don’t exist in isolation, instead they have the potential to be fulcrum points for learning in the communities areound them” (p.7) (complexity of systems) 

My understanding was further enhanced through a quote in an OECD guide for educators on LOs, where it is described as a: “multi-level concept involving individual behaviour, team work, and organisation-wide practices and cultures” and as a “place where the beliefs, values and norms of employees are brought to bear in support of sustained learning; where a “learning atmosphere”, “learning culture” or “learning climate” is nurtured; and where “learning to learn” is essential for everyone involved” (OECD, p. 2

Learning organisations

Considering the five disciplines inherent to LOs further:

It seems to me that there are some potential challenges with a blanket acceptance of LO concepts:

  • It reflects a cultural shift and shouldn’t be universally applied or the whole idea will backfire without discussion.
  • Structures of existing organisations may not be conducive to implementing change (ie their structure is too pyramidal)
  • The bigger the organisation the harder it would be to implement – it would be too unwieldy
  • Requires a long term view, which needs to be balanced against the short term needs for productivity and maintaining financial viability etc which naturally must take front seat at times.
  • Cost if implementing them in a structured manner

With reference to Senge’s 5 disciplines – to what extent is your organisation a learning organisation? 

I will briefly consider the faculty which coordinates the post graduate vocational training program with which I am primarily involved from the perspective of Senge’s disciplines.

By means of background when describing the extent to which the faculty is a LO, it is worthwhile to understand recent changes and challenges which have faced the faculty.

The faculty is under the auspices of a larger organisation (‘the college’) and until about 2007, the faculty maintained independence in its policies, procedures, finances and culture. Due to changes in management at the college level, this position was largely reversed and is now subject to the college’s policies, procedures, management structure, has lost independent management of its money and the culture has been eroded and replaced by one of cynicism and frustration. This occurred against a backdrop of an in-depth review of the particular speciality’s training program (Parker, 2015) with view to a potential significant change of the training program to be in alignment with other college training programs in response to recent changes to the practice of the speciality itself. Members of the faculty found it difficult to separate this question from the other issues facing the faculty and it also occurred against other controversial issues facing the management of the college as a whole, including a high profile lawsuit brought about by the college’s fellows (Komesaroff, Kerridge, Isaacs, & Brooks, 2015).

About three years ago, a new breakaway “society” was formed in an effort to claw back the culture and representation of the interests of the speciality, which perceived to be the greatest lost was with the changes that had occurred. Specialist training remains situated under the auspices of the college. This has increased the number of fellows occupying leadership positions (now spread across two organisations).

Senge’s disciplines will be considered from perspective of:

  1. Fellows and trainees of the faculty
  2. Committees of teh faculty
  3. The Faculty / college staff: as part of the bigger college
  4. The “Trainee of the Future workshop”

I feel I can make informed comments, being a past trainee, a current fellow, a member of organisational committees, and as a past employee of the college.

Much of my commentary will related to my observations in relation to the review of our training program that I undertook and follow up activities related to that. In response to my review of training, a workshop was held in September 2016, to which all fellows / trainees of the faculty were invited. It was facilitated by an external facilitator. Of the approaximately 600 fellows and trainees, about 25 were in attendance. I found this workshop a very enlightening and satisfying day, having been deeply involved with the review from the get-go, and felt quite satisfied with how it was run and the outcomes, which felt in alignment with, and true to the original project. Many of my comments will pertain to this workshop as it reflects the underlying philosophy and direction of our current faculty president, whose report from that workshop can be found here.

Change management and how it relates to Senge’s Discplines

Villa and Thousand (1995), as detailed further in Shortland-Jones, Anderson, Baker (2001) describe six ingredients for successful change, the absence of each one of which leads to a predictable issue. Shortland-Jones et al (2001) use this very useful framework to examine changes introduced in their educational academic setting at an instution in WA where a philosophical shift in education was being introduced. This simple framework was a useful approach for evaluating the change which had occurred, which what was observed occurred and a direction forward.

Three quotes from their article particularly resonated with me because they summarise the use of the matrix and the complexity of change and how intimately it is related to subtle shifts in culture:

  • “By listening and responding to the teachers and students as the program is implemented, flexibility is maintained, adaptability encouraged, and ownership achieved.” Pg 1.
  • “This matrix….has been used as a means of analysing, and in some respects evaluating change”  (final page)
  • “Change is as complex as culture itself” (final page)

In the table below, I have looked at the change components on which Shortland’s article is based and have tried to correlate them corresponding disciplines using Senge’s framework.

At times, I feel quite overwhelmed by the multitudes of models and frameworks that exist, each of which has some degree of overlap with others about which I have read. Examining commonalities between different models is one way of  to drawing out the common elements in order to understand the core concepts being identified, using different languages and terminlogies. Having looked at two frameworks this week (Senge and Villa and Thousand), it strikes me that while Senge’s disciplines relate to a learning organisation, they are also directly related to effective change management because learning (and therefore LOs) is virtually synonymous with change – change in perspective, knowledge, practice. To stop learning is to stop changing (is to stop living in many ways!) Below, I compare these two frameworks.

More change models than you can poke a stick at 

Incidentally, in Subject EDST5124, one of the assessments related to examining change in our educational contexts. In that essay, I enjoyed comparing and contrasting changes afoot at both UNSW and the postgraduate college I am involved with (Parker, 2016). In that essay, I briefly summarised four frameworks which I found particularly useful and informative and an image from that essay is reproduced below.


Leadership frameworks 

I wonder what teh collective noun for a group of frameworks is…perhaps it is a structure of frameworks. Nevertheless, in exploring Fullan and Ballew’s framework introduced in this week’s lecture, I cam across several others in a blog written by Webster (n.d.) including:

  1. Leadership that gets results (also known as Daniel Goleman’s golf clubs): uses concepts of emotional intelligence and identifies six leadership styles which should be used at the right time and in the right place to achieve good outcomes: affiliation, authoritative, coaching, democratic, pace-setting and coercive
  2. Situational leadership (Hershey and Blanchard): leadership styles are defined as one of four based on high and low scores on directive and supportive behaviours: directing, supporting, supporting and delegating
  3. Framework for leadership (Fullan and Ballew): described three personality characteristics (energy, enthusiasm and hope) along with five core leadership components: moral purpose, understanding change, relationship building, knowledge creation and sharing and coherence making.
  4. Agreement and certainty matrix (Stacey): examines proximity to certainty and agreement and looks at four states of change: simple, complicated, complex and chaos, and requires higher levels of interaction with those bringing about change and those affected by change (See below)
  5. Transformational leadership (Burns, Bass, and Kotter’s 8 step change model)

Agreement and certainty matrix (Stacy) – image from Webster (n.d.):


From these five models of change leadership, Webster neatly summarises the four features of successful change leadership as:

  1. Self awareness
  2. Self mabatement
  3. Social awareness
  4. Relationship management

Phew….I am just about structurally frameworked out now. So let’s look at the one which was actually discussed in this week’s lecture, and try to relate it to Chang matrix and Senge’s disciplining and apply it to the faculty conundrum previously discussed.

What can we learn from Fullan  and Ballew’s 5 part Framework for leadership? What are the implications for your workplace? 


Using Fallow and Ballew’s change leadership model, I will briefly consider how it relates to the change matrix and Senge’s discplines previously explored and will also examine features of leadership evident in the faculty and the changes it is currently facing as previously described in this blog:


Much food for thought here.

There are a lot of frameworks and matrices to evaluate organisations and change and leadership. I wonder how their relative value can be assessed and evaluated and how and if their core commonalities can be drawn out.



EDST 5126 Week 4 – Changing Curriculum and pedagogic design…..some things are timeless. 

I was first exposed to Biggs and Tang’s constructive alignment concept a couple of years ago and have personally found it, and parallel processes quite transformative in my thinking not only about education, but also my practice of medicine and life in general. Central to many aspects of life is having goals (intended learning outcomes). You then need to work out how you will achieve these goals (teaching and learning activities) and finally, you need to know whether or not you met the goals (assessment).


Rehabilitation Medicine practice: I run an day rehab program for people with diabilities at a major teaching hospital in Sydney. I now more explicitly think about the ‘intended learning outcomes’ (patient centred goals which are established between the patient, their doctor and their treating team which are SMART goals (specific, measurable, achievable, realistic and time-contingent). We then design ‘teaching and learning activities’ (therapeutic interventions in which the client participates eg medical treatment of spasticity with injections of botulinum toxin and / or oral medications, plus physiotherapy), and an ‘assessment’ (clinical review) is undertaken at the end of the therapy program to determine whether the goals were met. (Incidentally, something I am very interested in exploring further at some point is how principles of education can be applied to learning new skills in a rehabilitaiton setting…..maybe one for the future!)

Post Graduate Medical Education: in our four year post graduate vocational specialist medical training program, there is an assessment which is currently conducted during the first year of training. Historically, this exam was an “entry exam” into the training program to ensure that trainees had the requisite foundational knowledge and skills for the program: it was not part of the teaching and learning program itself, but an entry barrier. As it was not part of the training program, it did not have associated intended learning outcomes and associated teaching and learning activities.  Due to an evolution of the training program, this exam morphed from being a barrier entry exam to being situated within and as an element of the training program. The problem was that as this exam became an embedded assessment, no intended learning outcomes (curriculum) and teaching and learning activities were developed in parallel. I explored some of these issues further in a review of our training program I was employed to undertake (Parker, 2015).

This has caused angst for:

  1. Trainees who, rightly so, have voiced concerns such as: “how am I meant to prepare for an exam that is a component of a training program when I don’t know what the intended learning outcomes are (there is no curriculum) and there are no associated teaching and learning activities.
  2. Examiners: how can an assessment be set with confidence and authenticity when there is no curriculum against which it can be blueprinted? Although I personally am not responsible for this exam, it comes under the auspices of the Faculty Assessment Committee of which I am a member and I have an ethical issue with this assessment how it is currently situated and have advocated strongly for change.
  3. The accreditation body: the Australian Medical Council (AMC) is responsible for accrediting specialit training programs in Australia. While the misalignment, or more accurately the non-alignment of this examination with intended learning outcomes and teaching and learning activities has not been identified by the accreditation body (yet), this is a potential stumbling block for future accreditation of the training program and needs to be addressed.

Where to from here? The faculty and College are currently undertaking a review of entry to training standards. The fortunate timing of this process offers a natural opportunity to revise how this ‘entry exam’ is defined within the program, so that can be not only seen to be serving its proported purpose, but actually doing what it needs to be!

So, to sum up, constructive alignment is central to many aspects of life from my perspective….and while the contexts may change, the underlying principles of: what am I trying to learn? How am I going to learn it? And how will I know that I have learnt it? Apply. Such fundamental principles are almost universal and ubiquitous. 

Biggs, J. B. (2011). Teaching for quality learning at university: What the student does. McGraw-Hill Education (UK).

Parker, S. (2015). The Rehabilitation Medicine Trainee of the Future. Retrieved from https://www.racp.edu.au/docs/default-source/default-document-library/the-rehabilitation-medicine-trainee-of-the-future-report801e52afbbb261c2b08bff00001c3177.pdf?sfvrsn=0

EDST5126 – mid course review: Peer and self review – working together

While the prospect of reviewing one’s work with a peer is an inherently anxiety-provoking activity for me, like with many situations where we face our fears. reality was much better than anticipated, and in fact turned out to be an extremely valuable activity. 


Why was peer review effective?

  1. ATTITUDE: the peer with whom I was working adopted a positive, non judgemental attitude of openness with the intention of improvement not judgement
  2. STRUCTURED: both what we were reviewing and the framework being used for evaluation (the SOLO framework (Biggs & Collis, 2014)
  3. POSITIVE PEER PRESSURE: peer review in a non-threatening open-minded environment at this half way point in this subject was an effective motivational aid for me to maintain my progress towards the course requirements. While adhering to structured deadlines can be frustrating, it ensures that the work is done which is particularly important for students who face competing personal, professional and academic commitments.
  4. BI-DIRECTIONAL learning: I learnt as much by giving feedback as I did from receiving it.

What I learnt:

Discussion forums 

  1. Our group (me included) requested the discussion forums to be included in this course. With the benefit of hindsight, given the other media being used to capture work in this course (ePortfolia), maintaining the discussion forums too has added an extra layer of work in an already busy subject with required weekly tasks
  2. I have been inconsistent with posting on the discussion forums and need to schedule timely review of the discussions in addition to writing the weekly blogs…..if I comment in bulk, the relevance and inertia of the discussion thread is somehow lost and has petered out which means that the comments I make do not contribute to the ongoing discussion as the participants have already moved on.
  3. The discussion topics often go off in interesting directions (chasing rabbits down bunny holes)….while this is academically and cognitively stimulating I need to try to bring some of these discussions back to the question of the week.
  4. I should try to reference the academic literature in posts more


  1. While this has been strongly encouraged in other subjects in which I have enrolled, other than in the Foundations of University Learning and Teaching (FULT) course, I have not engaged with this format but have really enjoyed and can see the value in maintaining this to both fine tune my thinking but also as a means of capturing the artefacts of my work. It is also helping me to become familiar and comfortable with technology increasingly utilised by my students
  2. I really need to achieve a better balance between perfectionism and productivity when it comes to my posts, focusing on pithy, punchy posts
  3. I tend to use the blogs to capture new learning I have acquired in the week’s class (which is of value in and of itself for my future learning and reinforcing new concepts) as well as my reflections……in my final work submission, dissecting out the latter and increasing referencing to the literature in that will be important.
  4. Using the SOLO taxonomy, I am doing, while I do multiple unrelated points well and there are some unanticipated extensions, overall my posts lack the intermediate step of being ‘logically related’. This was also noted in my contributions to the discussion forums.
  5. Using Brookfield’s lenses (Brookfield, 2002) to structure my posts could be a means of focusing my work.
  6. I also need to incorporate comments and / or observations about how what I have learnt will change my thinking and practice.
  7. Despite these comments, it is good to observe that my work is evolving positively throughout the course.


I am really enjoying the class and the lively input from my peers and especially the organic discussions we have in class. The workload of polished weekly posts is proving to be quite challenging for me personally – keeping on topic, on task and on word limit.  While class discussions can feel somewhat free-range I have noticed that similar themes are brought up from week to week, for whatever the topic of the week is and thus, the key issues facing higher education are becoming increasingly clear to me as recurrent themes are seen to be spanning divergent topics.


  • Biggs, J. B., & Collis, K. F. (2014). Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic Press.
  • Brookfield, S. D. (2002). Using the lenses of critically reflective teaching in the community college classroom. New Directions for Community Colleges,2002(118), 31-38.

EDST5126 Week 6: Changing learning environments

The main learning institution I am involved with (the College of Physicians) has its main physical location in Sydney…..but unlike most HE institutions, there is very minimal teaching which actually takes place here – it is HQ for administrative. The college’s teaching is disseminated and occurs in hospital and other health settings. In my blog post this week, I explored other elements of the teaching and learning environment and took significant liberty with the term “environment”. I concluded that, despite the lack of physical learning environments, there are a lot of aspects of the other aspects of the “learning environment” that could be enhanced.


Environment Etymology 

My trusty online etymology dictionary tells me that environment is “a state of being environed” which is “the state of of conditions in which a person or thing lives” (from online etymology dictionary).

In subject EDST5122, one assessment task was to explore our own educational philosoph. My conceptualisation was of an educational tree which includes the elements and processes that influence an individual’s learning, metaphorically related to the environmental and other influences on a tree’s growth.

  1. Parker educational tree

These include:

  • Learning theories and frameworks
  • Global trends
  • Peers – the social learnign environment
  • Teachers – style, approach
  • Physical environment
  • Virtual environment
  • Political environment
  • Financial environment
  • Cultural environment
  • Educational resources such as equipment, technology, expertise


Weekly task – case study in educational environment

Identify strengths and weaknesses in your chose institution’s physical learning and teaching spaces. What possible changes to physical and/ or virtual spaces within the institutional resource constraints could improve student learning experiences? What planning should be considered for longer term improvements in your institution? 


  • Australasian Faculty of Rehabilitation Medicine – specialist medical training faculty under the auspices of the Royal Australasian College of Physicians (RACP)
  • 4 year training program consisting of 8 six month terms
  • Knowledge / skills are a combination of codified and tacit, including professional competencies
  • Educational model: Vocational training which incorporates elements of cognitive and traditional apprenticeships. Very limited whole group teaching –  only a monthly synchronous online two hour teaching session.
  • Approximately 250 trainees spread across metropolitan, regional and rural setting
Issue Observation Advantages Disadvantages Comment / suggestion
Geography Great diversity of clinical settings occupy the vast majority of “teaching and learning spaces”. ·       Lessons learnt from being exposed to a wide variety of settings could advantage all trainees ·       Other than ensuring minimum standards for site accreditation, difficult to standardise teaching environments 1.     Review of accreditation standards for sites in the light of rise of technology in medical education

2.     Capitalise on diversity of clinical experiences through informal sharing of experiences / insights by trainees

Teachers Teaching conducted on a volunteer basis by clinical supervisors. Faculty employs no educators ·       Clinicians have real world authentic experience, skills and knowledge ·       Educators have no formal training in education, potentially disadvantaging trainees.

·       The bulk of teaching is conducted by an enthusiastic minority. Risk of burnout

3.     Educational up skilling of educators

4.     Develop educational special interest group of supervisors with an interest in teaching

Technology education gap between supervisors and trainees, resulting in mismatch of knowledge / experience / comfort with using technology to support teaching. ·       Teachers can be taught by students ·       If new technology is rolled out, supervisors may be unable to support their trainees in optimally using this technology 5.     Develop pathways to support supervisors as new technology is introduced.
Current learning environments and activities Trainees have informal social media groups not under the auspices of the institution Greater freedom in self-expression away from the watchful eye of “big brother” No governance or moderation of behaviour / content 6.     With input from trainees, develop a suitably moderated and functional means of trainee interaction
Monthly bi-national training program – non compulsory two hour monthly teaching in virtual environment using “Vidyo” platform funning from February to November with a program which runs over three years. Trainees can attend at RACP offices in capital cities or via their own computer or asynchronously via recorded presentations ·       Didactic presentations

·       Frequent technology issues

·       No pre-set reading material

·       No available pre-set learning objectives

·       Limited interaction

Online training portal Some excellent resources available ·       Limited knowledge of these resources 8.     Marketing strategy needed
Online modules developed by the college Some excellent modules available ·       There doesn’t appear to be a structured plan with the modules that are appearing.

·       Limited knowledge of online learning opportunities amongst trainees and fellows. No Rehabilitaiton medicine specific modules. Unkow

9.     Evaluate effectiveness

10.  Develop further modules, having surveyed members regarding need

The way forward: 

Given the geographical distribution of training, there needs to be s strong focus on

  1. Cataloguing current online learning environments and resources
  2. Establishing where there are defects between current teaching and future needs
  3. Evaluating research, trends and directions in post-graduate medical education internationally to come up with evidence based recommendations
  4. Developing a priority list of actions and a plan for implementation

Learning environments in medical education – what does the literature say?

A brief scan of the literature identified the increasing role of the evolving eLearning environment as a key issue in meducation education with questions such as: how can materials be standardised? What about issues of confidentiality? Can “hidden tacit facets of physician education” (Hafler, et al, 2011, p. 440) be satisfactorily taught via eLearning? (Ruiz, Minzer, & Leipzig, 2006).

I found this article by Gordon et al (2000) very relevant to my particular learning setting. They performed a SWOT analysis of clinical teaching environments and came up with several suggestions for improvement and for focusing future work, and I was pleased to see that these were consistent with the suggestions I had come up with to address identified issues.


  • Gordon, J., Hazlett, C., Ten Cate, O., Mann, K., Kilminster, S., Prince, K., … & Newble, D. (2000). Strategic planning in medical education: enhancing the learning environment for students in clinical settings. Medical Education, 34(10), 841-850.
  • Hafler, J. P., Ownby, A. R., Thompson, B. M., Fasser, C. E., Grigsby, K., Haidet, P., … & Hafferty, F. W. (2011). Decoding the learning environment of medical education: a hidden curriculum perspective for faculty development. Academic Medicine, 86(4), 440-444.
  • Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic medicine, 81(3), 207-212.




EDST5126  Week 5 – Educational Evaluation


When you've finished changing, your're finished Franklin

Evaluation – what is it?

The etymology of Evaluation is from the French evaluer – in essence to extract the value out of something

Fox (2017) summarises three key elements of evaluation as:

  1. Assessing the merit / worth
  2. In a systematic manner
  3. With the purpose of making a judgement about the the worth of something, in other words its significance.

I find Burton’s simple three step process of evaluation (1970) very useful in exploring and conceptualising new concepts and new information.What so what now what

Prof Fox’s three point summary covers the first two elements (“What?” and “So what?”) well but seems to not capture so well the final step of “Now what? For me, evaluation must include that final phase of further action which may include change, or perhaps reinforcing the current state. Evaluation without action is interesting and perhaps academically rewarding but somewhat purposeless.

I have put the phases and elements of evaluation outlined in this week’s presentation / readings within Burton’s model

Evaluation what so what now what

What can be evaluated in HE?

Every aspect of higher education – people, processes and everything in between can be evaluated, which I have tried to capture in my image below.

what can be evaluated

Evaluation of impact of change is very important too and can be measured using Tilly Hinton’s IMPEL model, which we came across last year in EDST5124


What is in a tool?

A point which was highlighted in this evening’s class discussion is about the importance of evaluating the evaluation tool. This discussion echoed considerations of  evaluating assessment tools in education which I focused on in my assessments in EDST5122 last year. I have a particular interest in this area of formal assessment in education, being responsible for a high stakes examination for the medical speciality I am involved with.

The key elements of an assessment tool are summarised below. These are equally applicable when considering tools for evaluation in general.

  1. Reliable
  2. Valid
  3. Fair
  4. Feasible
  5. Cost effective
  6. Defensible
  7. Well-perceived
  8. Consistency
  9. Levelness

(Crosby, 2002 in Feather and Fry 2009 and QAA 2012)

These domains were expanded on through discussions this evening where the importance of considering these other areas was highlighted.

  1. Who designed the tool
  2. Who selected the tool
  3. The granularity of the tool
  4. The lens through which the results are evaluated: is it past focused or future focused?

Many of these aspects are highly relevant to the discussion underway about the tool being used to evaluate courses in HE ie moving from CATEI to My Experience. These issues were further highlighted by Uther (2017) in the class discussion forum where medicine courses across Australia were evaluated using various metrics from QILT from the perspective of both undergraduates and postgraduate students. These numbers could be used to support many arguments (“there are lies, damned lies and statistics”) and confounding factors (such as reputation of the university, numbers, culture, learning methods, teaching approaches and real world experiences.

Lie dam lies and statistics mark twain

Relationship between evaluation and research

Features          Research Evaluation
Purpose Produce generalisable new  knowledge Judges merit or worth of something specific
Why Conclusion oriented

= to improve

Decision oriented = to prove
How – methods Research tools Evaluation tools
Setting Often prescribed / controlled “Real world” environment
Directed by Researcher Party with a vested interest / stakeholder
Conclusions Make new research recommendations Recommendations are based on the question that was asked
Outputs Publish results / article Report for stakeholders
Evaluation of outputs Peer review Not usually peer reviewed


Features Journalist Academic
Depth More superficial Deeper
Independence from material Independent Vested interest
Relationship to truth Communicate the truth Discover the truth
Active More active More inactive
Product Writing Knowledge
Payment Usually independent of output Output determines future money (new grants)
Effect / impact  / next step Public opinion and funding Further research

May be reported by journalist

The discussions of comparing and contrasting research and evaluation and the approaches of journalists versus academics was interesting (and I have a distinct tendency to like putting ideas and concepts in boxes….it satisfies my mind’s need for order and control!). I struggled at first to understand why this exercise was undertaken……I felt that I had missed some nuanced point (perhaps parallels between these two comparisons with evaluation being more akin to journalists and research being more aligned with academics…..but this didn’t match wholly). I concluded that that the point of the exercise was by dissecting and breaking down the elements  to more deeply understand and appreciate and connect with the concept of evaluation, as it does have the tendency to have a rather nebulous ill-defined feel to it (much like the concept of governance!)

Why Evaluate?

Why evaluate slide.pngIf evaluation results in change and improvement, it most certainly adds value to an organisation……Evaluation must be planned for and systematic and occur at every level, from the big picture, right down to self evaluation, using a range of tools, as relevant to the things being evaluated. An organisation that does not evaluate will potentially either remain unchanged or change in ways which are not beneficial.

Trainee term evaluation in a medical speciality: a  case study

As an educational leader, if a reporter interviewed you about why your evaluation is necessary, how would you respond?

If the same reporter interviewed you about why evaluation can’t be done, how would you respond?

In responding to these questions, I have chosen to focus on the trainee term evaluation which is undertaken by vocational trainees in Rehabilitation Medicine of their six month terms. For the purposes of this scenario, as a leader within the training program, I am being interviewed by a trainee who is writing an article for the quarterly newsletter which is for fellows (graduates of the specialist medical program) and trainees.


 TRAINEE: Why do trainees have to complete the term evaluation every six months? It seems pointless because nothing ever changes as a result of it being done. It is a waste of time and effort.  

 SHARI PARKER (SP): The training program is evaluated in a number of different ways and from a number of different perspectives. These processes are multi-directional and have both formal and informal elements.

 At the individual level, trainees have continuous formative assessment by their supervisors throughout the term. Summatively, there are the various training elements you are very familiar with like exams, external training module essays and formal long cases, as well as your six monthly term assessment. Overall, these are used to determine your progress through the training program.

 Trainees get to evaluate the term placement, their work conditions, teaching, facilities and so forth using the trainee term evaluation which is what you are asking me about. I will talk a bit more about that soon.

 Supervisors undergo evaluation regularly too when they seek re-accreditation as a supervisor. This requires a combination of having attended mandatory training, as well as a process of self-evaluation.

 The training sites and terms are evaluated using a structured process on a three yearly basis by the accreditation committee.

From an organisational perspective, the assessment processes and training elements are evaluated: both the processes and the results which reflect on the training program as a whole. If the pass rates are low, this means that there may be an issue with the training program as a whole.

A couple of years ago, I was actually employed by the college to review and evaluate our training program. As a result of this process, a number of recommendations were made, many of which are currently being actioned. You can see my report here .   This was the first time such a review was undertaken and it took into account perspectives of trainees, fellows and reviewed objective data and incorporated local and international trends and perspectives. While this was a very important process, in my opinion, there must be a plan in place to undertake such an activity perhaps every five years, rather than such a project being initiated by one particularly insightful and visionary president. This holistic evaluation process must be independent of the incumbent leadership for there to be true growth and development.

From an even wider lens, the training program is evaluated regularly by the Australian Medical Council which is responsible for accrediting specialist medical training programs in Australia. They look at a range of elements and key indicators in the training program and make recommendations. You can see a bit more about their processes here

 The trainee term evaluation is a really important cog within this overall evaluation process, each element of which is vital.

Just so you can be familiar with what actually happens after you complete the form, the data is entered into a spread sheet by college staff.  The “new fellow” representative on the trainee committee is the person who is responsible for reviewing this data and addressing any significant “red flag” issues such as inappropriate behaviour or bullying.  In such instances, the new fellow directly contacts the trainee concerned and together they come up with a plan moving forward.

The new fellow provides a report to the Education committee annually and feedback is provided to individual training sites as part of their three yearly re-accreditation site visit so that any systemic issues or patterns that are evident can be addressed.

All feedback is de-identified which his really important because there are few trainees, and particularly in small areas, trainees could be identified. The aim is to both act on the information that you provide but also to protect your confidence.

So, to summarise, the term evaluation is a vital part of the process of evaluation for the rehab medicine training program as a whole and is also important for the individual sites. Armed with this information, it would be great if the return rate for these evaluations could increase so that meaningful information can be provided and acted on. I know that there is a degree of cynicism amongst trainees (and I have been in that boat too!) that the term evaluation is just a “ticking boxes” activity and is shoved in a dusty drawer, never seeing the light of day, but I can personally attest to the importance that this information plays in both improving the trainees’ experience and ultimately the quality of graduating specialists.

If you, or any other trainees have any specific questions about the process or any other aspects of evaluation within the faculty, I would be very happy to address them. Equally, if trainees have any suggestions about how we can improve the process of evaluation, we would warmly welcome them! This feedback is a key part of evaluating the evaluations…..a kind of meta-evaluation!

Decisions decisions decisions…..what to write about for the second assessment!

Evaluation – peer observation of teaching (POT)

A topic which grabbed my attention tonight is one which I may take further in the second assessment task for this course: that of peer observation of teaching in clinical medicine. I first encountered this idea in subject EDST5122 where we were required to evaluate some examples of teaching. This was the first time I became aware of the process of POT and i recognised that, while this is an evolving area of HE which has the potential to improve teaching and learning, that it is largely absent from clinical teaching. That being said, POT is not without its risks, including potentially retrenchment or denial or promotion, changing the culture of an organisation and fostering cynicism of staff towards leadership and also whether this will actually achieve what it is set out to.

One potential outcome I superficially explored in my essay was how POT could be introduced in my own clinical context, potentially developing a template for it to be rolled out in my department, the clinical school at which I teach and possibly further afield. This evening’s tutorial has cemented this interest and it is an area I may delve further into in this subject in the second assessment.

Reflection……..a self evaluation!

What: In writing blog posts for this course,  I tend to try to do too much, resulting in long meandering posts which far exceed the word limit which take up a lot of my time. I try to cover too many areas in too much detail. I lose the woods for the trees. As a result, I am getting behind with the class work for this course.

So what?  The impact can be seen on a number of different levels:

  • Academic: there are rules (albeit flexible) with regards to word limits. If I don’t edit my work substantially, it will be outside the
  • A short punchy focused blog post will have far more impact: firstly because it is more likely to be read and secondly because any valuable ideas will be more clear and not lost within the noise of my logorrhoea.
  • Personally, spending too much time on posts is contributing to a sense of increasing overwhelm and stress. If I don’t get on top of this, there are potentially negative implications for me personally and academically.

Now What?

  • As I catch up with the blog posts I am behind with, I need to learn to drill down and distill the key elements of an issue, to be able to describe them succinctly and communicate them efficiently.
  • I will, with discipline, set a time limit for writing my blogs and stick to it, no matter what!

(note to self…..this reflection is 239 words and not too far off how long I should aim to make my blog posts!) 



Borton, T. (1970). Reach, teach and touch. London: McGraw Hill. Carper, B (1978). Fundamental patterns of knowing in nursing. Advances in Nursing Practice1, 13-23.

Fox, B. (2017). Session 5: Evaluation (PowerPoint presentation). Retrieved 2017_03_28 from https://moodle.telt.unsw.edu.au/pluginfile.php/2506495/mod_resource/content/1/EDST5126_S5_2017.pptx

Feather, A., & Fry, H. (2009). Key aspects of teaching and learning in medicine and dentistry. A handbook for Teaching and Learning in Higher Education, 424.

The Quality Assurance Agency for Higher Education (QAA). (2012). Understanding assessment: its role in safeguarding academic standards and quality in higher education. Retrieved 2017_03_28 from http://www.qaa.ac.uk/en/Publications/Documents/understanding-assessment.pdf

Uther, P. (2017). EDST5126 Week 5 class discussion forum. Retrieved 2017_04_04 from https://moodle.telt.unsw.edu.au/mod/forum/discuss.php?d=514124