RSS Feed

Posts Tagged ‘research’

  1. Education – my thinking

    12 November 2018 by shartley

    I was recently challenged by someone to add more of my own thinking to this blog, given that recently it has been more about other people’s voices. So sitting in my gorgeous hotel in Bath, occasionally looking out the window (view in image above) for moments of contemplation, I have bashed out where my head is currently at re education and my PhD. By the way, for my more general thoughts, less education related, I have a writing blog too. Often the lines blur so the decision as to where to post can be quite the quandary.

    So here goes.

    I am passionate and quite emotionally tied up with:

    1. Students’ gaining agency through education so they feel empowered to make choices and decisions about their own lives and believe they can have an impact on the people, communities and societies around them.
    2. Creating a broad curriculum that students value and engage in.
    3. Implementing pedagogy that enables students to value and engage with the curriculum.
    4. This generally means the curriculum needs to be relevant to real life in terms of what is being learnt, how it is learnt and how it is assessed.
    5. Also means that there is a need to not only focus on students’ attainment on knowledge but also their development of skills, attributes, competencies, capabilities, and other closely related terms.
    6. Breaking down the restrictions, barriers and the risk levels teachers feel, to enable students to learn and acquire the knowledge, skills and attributes they need now and into the future.

     

    Thus, there are several areas in education that really make me angry and frustrated.

    1. Teachers who just want to deliver information to compliant students.
    2. People outside the profession who think education is merely about delivering information to compliant students.
    3. The lack of recognition that to do more than deliver information takes time and energy.
    4. The teachers who think their innovative teaching method is the one and only way. Teaching needs to be fit for purpose.
    5. Schools that promote themselves as being a “PBL school” or some other particular method of learning annoys the hell out of me. Teaching needs to be fit for purpose.
    6. Students and their families who focus on the final grade at the end of over a decade of education and not appreciating learning for its own sake. This is exemplified by students who say “just tell me what I need to know”.
    7. Schools and teachers who cater to students and families’ single-minded focus on grades and/or want to maintain an outdated image of compliant students sitting in rows, working in silence.
    8. The confines of prescribed curriculum reducing teaching to a tick-box approach to covering material.
    9. Curriculum that claims in overarching statements at the front of documents that they are achieving a range of knowledge, skills and attributes through that curriculum and then in the back end, reduce teaching to be mainly about the mere delivery of knowledge.
    10. Dichotomies in education that reduce concepts to either/or concepts. I am sometimes guilty of buying into some of them in my research and writing. I am particular against the ideas of traditional versus progressive teaching and knowledge versus skills and attributes. Again, it’s about teaching that is fit for purpose. Fit for the knowledge, skills and attributes being sought, fit for the students undertaking the learning and fit for the context and available resources.
    11. The terms “non-cognitive skills” and “soft skills”, as if leadership, creativity, interpersonal skills and so on, do not require thinking and hard work to develop and improve.
    12. The term “21st century skills”, as if these skills have only been needed for the last 18 years and not before. How ridiculous!
    13. Socio-economic inequities in education. I have worked in four private schools with various level of fees and funds to expend. My own children went to private schools for the most part. Despite my own actions and behaviour, I desperately want public schools to be better resourced. Technology adds to the socio-economic gap in education and yet I am still an advocate for technology in teaching because it is such an integral part of our lives. I worry how much education adds to income inequality in our society.
    14. Other inequities in education that occur on the basis of gender, race, learning difficulties and much more.
    15. STEM (Science, Technology, Engineering and Mathematics) currently being valued so much more than the Humanities. Is it really to cater for a job market (and thereby treating education as the mere creation of human capital)? Or is it about reducing the amount of critical thinking about societies, communities and the people within them?
    16. The treatment of education as if it is about producing human capital. I will acknowledge there is an element of that occurring because it is inescapable in a capitalist society but there is so much more to education than this!

     

    This thinking informs the research I want to undertake for my PhD.  It has led me to Enterprise Education which has the potential to encompass all that I desire in education. I also resist the wholesale embracing of capitalism and therefore would prefer if Enterprise Education did not overly feature entrepreneurship and financial gains. To me, Enterprise Education is about developing students’ enterprising skills and attributes in the broadest sense, allowing all students to flourish. I want to discover the best objectives, curriculum, pedagogies and assessment processes in Enterprise Education that is conducted at a scale beyond the classroom, at a whole grade level, preferably more than a one-off experience but a whole year program. I would love to just submerge myself in a few of these programs and analyse what works best.

    Attending the ISBE Conference last week was a fantastic immersive experience in the thinking of a whole bunch of amazing people working in this area but it also added to the imposter syndrome monster within me. My struggle is the more academic aspect of doing what I want to do. I need to work out theoretical approaches/lenses and frameworks. This is what keeps me awake at night and occasionally drives me to tears. Am I ever going to get my head around this stuff? Am I ever going to be able to produce good quality academic work that can make a difference in education by influencing curriculum and helping teachers to happily do the best for their students? Well, that’s my dream. I’m sure once I have a grip on the academic theories I will be able to analyse and write and have a voice in the arena but right now there’s a huge wall in front of me. I have ten days to write a decent PhD proposal, including a literature review, while I also enjoy the pleasures of being a tourist as I finish up in Bath, go to Prague, and then head home. I know, tough life and all. I will submit a proposal at the end of this time but it won’t be anywhere near the quality I desire unless an epiphany occurs.

    Soon this initial hurdle will be behind me and I won’t have to think on it until the PhD truly commences in 2019. In the meantime, back home, I’ll be preparing for Christmas and finding somewhere new to live (long story). I am looking forward to hosting a party for my fellow post-grad education researchers, visiting family in Adelaide, going to the cricket and tennis, reading for pleasure and experiencing the glorious heat and sun of summer in Sydney, Australia.

    PS My playlist (shuffled) as I wrote this post.


  2. Risks, Fears and Concerns: Issues teachers have with pedagogical change

    6 February 2018 by shartley

     

    The last few days I’ve been doing a slow dig into my research. I am looking at the concerns teachers have about shifting to more student-centred pedagogy. I am also trying to find out what can be done to alleviate these concerns, particularly in the form of professional development.

    For non-teachers, very roughly, pedagogy is the art and science of teaching, the methods used in the teaching and learning process. Student-centred learning is when students have more of an active role in their learning. Instead of merely being recipients of information and drilled into skills, they undertake activities that develop skills in critical thinking, problem solving and much more. By having greater choice over the process of learning, students often take on more ownership of it.

    A traditional view, reinforced by popular culture, is that learning at high school consists of students sitting in rows, in a classroom, with a teacher talking at them, leading a discussion or showing some boring film. Sometimes learning needs to be with a teacher directing a class from the front but often it is not the best way to learn. It is probably the best way to maintain order and control in a classroom. Complete order and control is not necessary for learning to occur. Structure and purpose is important most of the time, but not so tight it confines students’ minds from thinking for themselves.

    Many teachers have difficulty letting go of order and control. In their minds, quiet work is productive work, and reflects well on them as teachers. It is so incongruent of a concept, it is hard for them to learn and implement strategies that allows students more freedom because the perceived risk is too high. One of the articles I’ve read about this is called ‘What if students revolt?’. This happens more in the higher grades because students have so much pressure on them to achieve in high-stakes tests, such as the HSC, they just want to be told what they need to write to do well in the exam, having very little focus on learning itself. I’ve had one frustrated student who pleaded, “Just tell me what I need to know”. Memorising and regurgitating information just doesn’t cut it anymore. The fear is also about students not achieving as well in assessments and that also reflecting badly upon the teacher. It is being scared of looking a fool by trying something out of their comfort zone and possibly failing.

    Textbooks give the false illusion that learning is linear and straight forward, a mere comprehension task. Textbooks also help teachers know exactly where they are up to in the teaching process and can check-in with other teachers about their pacing. It shouldn’t be about where students are up to in the textbook but where they are up to in their learning.

    This fear is exacerbated if teachers feel they don’t have the backing of the school leadership or the wider school community (eg school board, parents) and are constantly slammed in the media and by government figures. The literature calls this a lack of relational trust. From my  own experience, this can occur in several ways. For instance, there is the fear that school leaders will change their mind and switch back from the new innovation to the traditional way or leapfrog onto something else again, thereby wasting a lot of time and effort of everyone involved. Sometimes when a teacher tries a new teaching method, as advocated by the leadership, and it results in complaints from students and/or parents, the leadership kowtow to the complainers when the teacher should be backed by them. It is demoralising and makes teachers even less inclined to change what they have always done. The risk to change becomes way too high. Some schools have gone through incredible changes, only to have a new principal come in and change it all back again.

    Most schools are going down the student-centred learning path but veer back to traditional teaching for the HSC when marks trump real learning. It’s the HSC game really. I hate how many times I say to my classes that in reality ‘xyz’ is this but for the HSC it is that. For example, in HSC Business Studies students have to write a business report. They are usually taught to write in exams an Executive Summary as a substitute for an Introduction and to keep it as brief as possible. It is purely to tick the box that they did one, and move on as quickly as possible to provide content that will produce the bulk of the marks. In reality, executive summaries actually are what they sound like they should be, a summary of the report and a rule of thumb is that it should be a page in length. I find this so frustrating! Exams are so removed from real life but are a convenient way of ranking people, distributing them across some statistically desired graph. We supposedly have a standards based system, meaning students are assessed against standards instead of against each other but when it comes down to it, results are manipulated to reflect a particular norm. A friend who has marked Food Technology exams has reported that when the statistics revealed the majority of students failed a question, the markers had to go back and scratch for just a word or two that could just possibly mean, perhaps, the student may have actually understood what they were writing about in some way.

    An issue with all this is the conflicting demands placed on teaching and learning. The national curriculum’s general capabilities provides a good overview of what school education should be about:

    • Literacy
    • Numeracy
    • ICT capability
    • Critical and creative thinking
    • Personal and social capability
    • Ethical understanding
    • Intercultural understanding

    However, NAPLAN, the only compulsory external testing system in NSW before the HSC, is only about literacy and numeracy. Since NAPLAN and HSC results are so public, schools can fall into the trap of teaching to the test, resulting in formulaic responses that produce solid results. Teaching for genuine, deep learning, is much harder to test properly so that in our current system, real learning can result in wildly inconsistent outcomes in NAPLAN and the HSC. I always professed I wanted my children to be motivated more by learning than by assessment results. Yet I had to compromise that ideology when Emma wanted to learn Economics for the HSC but performed better in Ancient History, which she had been interested in for years but was now bored. To achieve the mark she needed for her desired university course, the interest in learning was traded in for a higher mark.

    It takes time to implement change. It takes time to learn new ways of doing things. It takes longer to do something for the first time than it will subsequently. The planning time for new pedagogical practices will be longer because the style of learning is new and taxes the brain harder than just doing what was done before. Many teachers are used to Heads of Department writing programs and handing them over to be followed (or not). Now it’s more about collaborative planning and coming up with new ways to work together as a team of teachers. It feels like independence is being stripped away as well as authority in the classroom. However, it actually should be giving teachers greater ownership and pride in what they do because they design the learning process instead of deliver information. By not being up the front of the class so much, teachers should also have more time in class to have one-on-one discussions with individual students about their progress

    Technology has a lot to do with the shift away from traditional learning, enabling students to participate in the learning process in new, innovative and fun ways. Just watch how young people jump to YouTube for tutorials in how to do something, from knitting (I have seen a student do that with my own eyes when they were supposed to be working on something else) to changing a tyre. The Internet is their go-to for communication, information, entertainment and well, everything, really.

    Many teachers find it hard to understand and adopt technology and are fearful of it. They are already forced online to mark their rolls, maintain a grading system, complete their welfare reporting and much more, so that to also have to learn more technology for the teaching process, it can be overwhelming. Technology can also be unreliable in schools, being such an expensive commodity. To me, the gap between the technology haves and have nots is the biggest divide in education of modern times. My son gained 17 marks on his school assessment in his Information Processing Technology HSC exam with a great deal of help from Eddie Woo’s YouTube videos. If we didn’t have a reliable internet at home this would not have been possible. I worry about students who don’t have access to reliable Internet at home – it will hold up their education and their adjustment to participating in life, at work and play. Schools need to help out more in this regard, but that’s a completely different post to write.

    I know and recognise I have been a bit on my soapbox in this post. I partly wrote it here to shake it out of my system so I can approach these issues in a more academic and studious way for my research. All these concerns are real and need to be acknowledged and addressed. I am looking forward to investigating how this can best be done.

    Further Reading

    Boschman, Ferry, McKenney, Susan, & Voogt, Joke. (2014). Understanding Decision Making in Teachers’ Curriculum Design Approaches. Educational Technology Research and Development, 62(4), 393-416

    Dole, S., Bloom, L., and Kowalske, K. (2016). Transforming Pedagogy: Changing Perspectives from Teacher-Centered to Learner-Centered. Interdisciplinary Journal of Problem-based Learning, 10(1)

    Le Fevre, D. (2014). Barriers to implementing pedagogical change: The role of teachers’ perceptions of risk. Teaching and Teacher Education, 38, 56-64

    Siedel, S. B., & Tanner, K. D. (2013). “What If Students Revolt?”–Considering Student Resistance: Origins, Options, and Opportunities for Investigation. CBE – Life Sciences Education, 12(4)

     

    * This post is also on my general writing blog: https://squibsandsagas.blogspot.com/2018/01/risks-fears-and-concerns.html


  3. Mind the Gap: Research and Schools

    1 November 2016 by shartley

    217048717_08e3e922f1_m

    Image by Mikel Ortega at https://www.flickr.com/photos/mikelo/217048717

    Last week I attended the AIS Education Research Symposium.  When I worked at Oakhill College I was part of the team that created and implemented REAL (Relevant Engaging, Active Learning) and then we researched its impact.  Unfortunately I left Oakhill before the end of the two year research project but thankfully I remained involved to an extent.  We designed REAL to create a paradigm shift in the teaching and learning environment at the school through a transparent curriculum in a detailed but clear (student-friendly) format, structured for students to know each lesson or week:

    • What is to be learned
    • How it is to be learned, and
    • The evidence of successful learning

    The research project examined the impact of REAL on student outcomes but effectively discovered the effect it had on teachers too.  The research revealed REAL as a significantly successful program, although teachers found it hard to adapt their practices, and I’m proud to have been associated with it.

    But this post is about the symposium.  The main participants at the symposium are from schools that have received a funding opportunity from AIS and are at the start, middle or end of a two year research project.  It is also for any other teachers who are keen to make stronger links between research and teaching practice.  And the first keynote speaker, Dr Stacey Waters, was exactly on this topic.

    It is extremely hard to make a cultural shift in any institution, let alone a school which has so many entrenched perceptions of what it should be like.  Nearly everyone in the world experiences school and our experiences are reinforced by pop culture’s representation of it, this image being a teacher up the front instructing students sitting in rows, resisting the learning process (except when they have a superstar teacher like Michelle Pfeiffer or Robin Williams).

    This is not the one best way to learn.  Academic research has indicated this repeatedly, yet many schools resist changing (much like their students resist learning).  I think part of it is the conflicting voices dominating the discourse of how schools should operate. As mentioned before, there is the entertainment industry image but there is also the political viewpoint that schools must do better, meaning better at publicly published score achievement (PISA, NAPLAN, HSC).  In education conference circles there are a few (mainly men) who have a certain popularity, often stringing out their one concept stories for decades (I’m thinking the hole-in-the-wall story and the idea that schools kill creativity).

    The academic voice is often lost in all this.  There are some who are picked up by politicians and/or the media. For example, John Hattie and his Visible Learning ideal is lauded by politicians and the media.  However, even the John Hatties often remain ignored in the day-to-day classroom, for a number of reasons.

    Personally I think it stems from a dominating need of isolated teachers in their classrooms to have control and order.  Control and order is easier if learning is considered to be the mere accumulation of knowledge.  It casts a dark shadow on education.  Teaching is better if it is collaborative, learning is better if it is collaborative, but it is much harder to maintain control and order in a collaborative environment when there isn’t a desire to learn in the first place, by teachers or students.  It is harder to know what individual students are doing in a collaborative environment and if it is known that they are not participating in the learning process, it is hard to know what to do about them.  There are plenty of theories, methods and systems that can be put in place but really, it all comes to dealing with individual motivations and desires to learn.  And to be honest, the better learning environments take more time and energy to plan and monitor.  I love an active collaborative learning environment in my classrooms but boy, it’s hard work a lot of the time.  

    Prof Ruth Deakin Crick shared this quote as part of her presentation about the use of technology to change school culture from exam-driven teaching to a learning culture with dynamic pedagogy and engagement (image, however, was clipped from elsewhere):

    screen-shot-2016-11-01-at-6-05-25-pm

    Teachers are so busy and comfortable with what they know well and what they’ve been rewarded for in the past, they don’t want to become even busier and take risks that result in lost control and disapproval.

    So now we can see why some resistance to change exists, let’s return to the academic voice and other reasons why it is difficult to implement research findings into school practice.

    To be able to research in a detailed, scientifically credible manner, academics focus on narrow areas to reduce multiple confounding factors in their study. They generally want to discover a cause and effect, and the size of that effect, but to be specific and certain they can only concentrate on one cause and one effect at a time, whereas education is much much more complicated than that.  It is like reducing economic modelling to two products in the market.  It illustrates a concept but an economy with just two products is not real life.  This is why Hattie’s study is so appealing, it throws hundreds of studies into the mix and calculates which causes have the most effect.  However, one point worth noting, is how long this all takes.  A research project can take years, and then months to write and publish in an academic journal, and then even longer to garner the attention of those who it really affects, people in schools.  I’m not sure of the age of the studies Hattie includes in his meta-analysis.  Due to the narrow focus and time-lags the question was raised at the conference as to whether the academic rigour should be sacrificed, to an extent, to allow more pertinent and practical studies be undertaken in a more timely manner.  I don’t know the answer.

    The publish or perish pressure on academics is a significant factor too.  The money associated with grants often comes with a proviso of what must be researched. Dr Stacey Waters also referred to the Excellence in Research for Australia (ERA) system which awards points according to the category of grants.  To be published, it helps if the research is something new and different, rather than going deeper into something that has already been examined.  The dream situation for an academic though, is selling books.  And books sell if they have something novel and easy to implement.  We all know how hard change management is, but just how many books advocate just ‘x’ number of steps (eg Kotter’s 8 steps) like it is a simple linear process?  It’s not.

    Even if academics find something that is commonly considered important to implement in schools, the process is slow and usually ineffectual. Researchers are already balancing teaching and research and then they need to market their findings so that schools will take their ideas on board, when so many ideas are already present.  Dr Stacey Waters says that publishing and even training people is not enough. One of her presentation slides said:


    Implementation is most successful when…

    • Practitioners receive training and coaching
    • The organisation provides the necessary infrastructure for training and coaching and regularly evaluates
    • The community is fully involved in the selection and evaluation of programs

    (Fixsen, Naoom, Blasé, Friedman & Wallace, 2005)


    She argued that schools should build relationships with universities and establish a research culture by providing easy access to academic journals and having a forum to discuss what these journals have to say.

    Before Oakhill obtained the research funding opportunity from AIS, we asked some universities to help us perform academic research.  We discovered that not only did they want to be paid consultation fees, that some even wanted to virtually take over the whole process.  We backed away quickly and were relieved and grateful to receive the AIS funding opportunity.  As part of the inaugural recipients of research funding there was only minimal guidance but still, some important structural requirements, including an academic advisor.  However, we ended up only spending 50% of the budgeted amount we allocated for the advisor. He was extremely helpful for implementing research that produced quantifiable data and how that data could be used, but partly due to how late we were and partly due to the lack of need, we didn’t use as much of his services for the writing part as we thought we would.

    So here we were at a conference that was bringing together academics and teachers, albeit those already interested in research. As Terrie Jones tweeted, “Teacher practitioner researchers in partnership with academics bridging the research practice divide? This room. #AISRandD16”.  One of the sessions I enjoyed the most was about Case Study as a form of research.  From what little I have been involved in research at a tertiary level, I have found case study methods frowned upon because it is not appropriate to extrapolate something that works in one circumstance as being applicable to a wider range of circumstances.  Dr Kimberley Pressick-Kilborn argued that yes, three case studies are better than one, but much can be learned from them and proceeded to demonstrate how powerful they could be but also recognising their limitations.  I tweeted that I’d love to work on a case study with Dr Kimberley and she responded with a “Let’s talk” which I will probably take her up on some time next year (see screen grab below).  However, I’m not looking to do research like this until at least 2018.  I started a Master of Research (MRES) at Macquarie University this year but stopped just a few weeks in due to lack of time. I’m currently not working full-time so I can help my son through his HSC (he has autism and anxiety issues).  I considered studying also during this year ‘off’ but I need to make him the priority, not my own work. After that I’ll be open to all sorts of projects!

    screen-shot-2016-11-01-at-6-30-03-pm

    The people at this conference are the core people of their organisations interested in learning and continuous improvement.  What we actually need to learn as researchers within schools, is how to gather-in colleagues to a learning mindset and join with us.

    When we first attempted researching the REAL project ourselves, we applied to present at academic conferences in Seville, Spain, and New York, USA, and were approved for both based on our ‘abstracts’.  This was before the AIS research funding program even existed and we were without any support from universities.  We then had to write papers for these conferences, the New York paper being subject to a peer review.  It failed one reviewer but the other advised us to change the format significantly, which we did, and it was then published <http://shanihartley.cgpublisher.com/product/pub.264/prod.58>*.  We also had to review other papers which was a great learning experience for us.  At these conferences there was much said about education at all levels but over 90% of the presenters were from universities.  Universities seem to take a more hit-and-run approach instead of working in tandem with the schools they research.  It would be nice to see or even experience more collaboration between schools and universities in the research process at academic conferences.  Perhaps we were better off without a university’s own agenda. We were unusual in being school teachers conducting and presenting our own research.  There should be more of it.  

     

    * Despite numerous emails to the publishers, the bios of the authors were not corrected.
    This post is also found at http://squibsandsagas.blogspot.com.au/2016/11/mind-gap-research-and-schools.html

  4. Unit evaluation – composing a questionnaire

    5 April 2015 by shartley

    As part of my Masters of Education at Macquarie University I am studying EDCN800 Introduction to Educational Research.  Yesterday’s blog post lamented the frustrations I was having with the statistical aspects of the course. Today I was looking at the construction of questionnaires and the exercise set was basically the writing of some questions for evaluating a unit of study.  The thinking behind these questions is a little influenced by John Hattie’s Visible Learning (2009) which is currently guiding a research project with which I am involved, but mainly they come from an accumulation of experience and study of a wide range of material.  The areas I’ve noted and the questions themselves are not even close to being exhaustive lists. I actually hope to later develop this into a real questionnaire to evaluate my own courses.  I’d be interested in knowing what you would include.

    Aim: Evaluating a unit of study within the Masters of Education program.

    Research participants: Students undertaking the unit of study

    Key areas:

    The learning environment

    • Online
      • Ease of understanding/following procedure and instruction
      • Allows for communication and interaction with instructor and peers
      • Feels inclusive of all (gender, disabilities, internal/external students, etc)
    • On campus
      • Inviting environment (temperature, lighting, seating comfort, etc)
      • Allows for communication and interaction with instructor and peers
      • Feels inclusive of all (gender, disabilities, internal/external students, etc)

     

    The curriculum

    • Knowledge, understanding and skills
      • Made clear
      • Covers a range of lower-order and higher-order thinking skills
      • Challenging but achievable
    • Relevant
      • Builds on prior knowledge
      • Practical application to workplace and/or experience
      • Includes contemporary issues
    • Assessment
      • Both the assessment and related criteria are easy to understand
      • Aligns with goals for knowledge, understanding and skills
      • Offers choice
      • Fair, equitable and achievable (task and weighting)
      • Provides opportunity for feedback during the process
    • Transparent
      • Purpose of the learning is clear
      • All elements/tasks of the course are clear and upfront
      • What lies ahead in the course is clear
      • Timing is clear (module lengths, due dates, etc)

     

    The teacher

    • Knowledgeable
      • Course content
      • Student needs
      • Teaching methods
      • Contemporary context
    • Communication
      • Approachable/personable
      • Quality feedback
      • Clear in expectations
    • Strategies
      • Variety
      • Engaging
      • Motivates
      • Relevant
      • Easy to follow/do
      • Involves collaboration (at times)
    • Flexible in approach
      • Adapts to the learning needs of students
      • Adapts to changing circumstances

     

    The impact

    • Students know and understand more and can do more as a result of the course
    • Students are engaged, interested and/or enjoy the course
    • Students increase their desire to learn
    • Students obtain a sense of achievement from completing the course

     

    Examples of research questions to be included in a questionnaire conducted at the conclusion of a unit of study.

    The learning environment

    To assess the experience of the student in using the online page I would include these questions:

    The online course page was easy to navigate.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The online course page clearly presented the unit’s requirements.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The online course page supported collaborative learning.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    To conclude the online environment section I would ask an overarching question such as:

    How satisfied were you with the online environment in this unit of study?

    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied

     

    Knowledge, understanding and skills

    To assess the experience of the student’s learning process I would include these questions:

    The course outcomes were easy to understand.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The course was challenging.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The course extended my understanding of the topic.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The course helped me to think more deeply about the topic.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    To conclude the knowledge, understanding and skills section I would ask an overarching question such as:

    How satisfied were you with what you learned in this unit of study?

    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied

     

    Assessment

    To assess the student’s experience of assessment I would include these questions:

    The assessment requirements were easy to understand.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The assessment criteria was easy to understand.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The assessment criteria matched the assessment requirements.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The assessment requirements were challenging.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The assessment process extended my understanding of the topic.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The assessment process included helpful feedback before final submission.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree
    • Don’t know

     

    The assessment process was fair for all students.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree
    • Don’t know

     

     To conclude the assessment section I would ask an overarching question (or two) such as:

    How satisfied were you with how assessment was marked in this unit of study?

    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied

     

    How satisfied were you with the process of completing assessment in this unit of study?

    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied

     

    Communication

    To assess the student’s experience of the communication process I would include these questions:

     The teacher was easy to understand.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

     The teacher provided clear expectations.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

    The teacher provided helpful feedback.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree
    • Not applicable

     

    The teacher was approachable.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree
    • Not applicable

     

    The teacher provided timely responses to questions asked.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree
    • Not applicable

     

    To conclude the communication section I would ask an overarching question such as:

    How satisfied were you with how the teacher communicated in this unit of study?

    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied

     

    The impact

    To assess the impact of the unit of study I would include these questions:

    I know and understand more about this topic area as a result of completing this unit of study.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

     I found this unit of study interesting.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

     I want to learn more about this topic area.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

     I obtained a sense of achievement from completing this unit of study.

    • Strongly disagree
    • Disagree
    • Neutral
    • Agree
    • Strongly agree

     

     To conclude this section I would ask an overarching question such as:

    How satisfied were you with what you learned in this unit of study?

    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied

     

    Construction of the Questionnaire

    These questions were designed to align with what I thought was most important for assessing a unit of study.

    They were written to be easily understood (“natural and familiar language” with “clear, precise and relatively short items”, Johnson & Christensen 2014, p.193) and allow for an appropriate range of options.

    To keep the questions easy to follow I used a fully anchored scale for all questions and there were only two styles of rating scales: (1) Agreement – for students to assess the elements of the course (2) Satisfaction – for students to assess the impact of the course on themselves in a broader and more personal sense.

    On occasion an option of “Don’t know” or “Not applicable” was added to allow for students who had not experienced that particular aspect of the course.

    The number of points were kept to five to assess students’ ambivalence and avoid irritating participants by forcing a stance.

    The wording was kept consistent where possible between sections, such as asking if the course/assessment/convenor were easy to understand, to allow for direct comparisons between the elements being researched.

    I believe I have avoided leading or loaded questions although at times I was tempted to add an adjective or two which would have broken this principle.

     

     

    REFERENCES

    Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement.  London: Routledge.

    Johnson, R. and Christensen, L. (2014). Educational research: Quantitative, qualitative, and mixed approaches. (5th Edition). Thousand Oaks California: Sage.


  5. Research

    4 April 2015 by shartley

    20150404-220801.jpg

    Image source: author’s own textbook

    I’m currently studying EDCN800 Introduction to Educational Research at Macquarie University.  It is the only compulsory subject in my course but I put it off to last because it seemed so dry, and well, boring.  I have my regrets, however, since it would have been quite useful to know what I should be doing before I submitted abstracts on behalf of my team to international conferences.  When we were accepted to these conferences I had to write academic level articles on the basis of haphazard and amateurish research.  One of these articles was for a peer-reviewed journal and one of the two peers who assessed the paper slammed it for not being written in the acceptable academic format.  I had avoided the more academic format because I didn’t want to pretend that the research was formally conducted.  I have now resubmitted the piece into a more acceptable format but it still awaits final approval.

    More recently, I have been trying to support a friend who has been designing real proper research under the guidance of a university professor.  The professor’s critiquing of the attempts to write a research question and plan the research methods was a painful process but the frustration was worth it in the end because I think there is a very valuable research project currently underway.  As I do this course I can, in retrospect, see more clearly what was required and if I had completed this subject before this year it might have been a much less painful process.  Now, as I study the ethics of research I wonder how much more should be done to cover ethical considerations in my friend’s research.  It is also giving me more depth to my knowledge of research methods for when I teach Society and Culture.

    In the first semester of my Masters of Education I chose one subject (curriculum) because a friend was also doing it and another subject because I felt knowledgeable in that area (pedagogy).  I had enrolled in the course just so I could obtain the piece of paper and letters to look good on my CV but within a couple of weeks of participating in these two subjects I was enjoying myself immensely and did quite well as a result.  However, the one aspect that I was continually criticised about was the negligible evidence to support my (soapbox) statements.  I have improved a lot in this area since then.

    Now as I study EDCN800 I expect high achievement from myself but I’m not succeeding. I’m engaged in the subject because of the afore-mentioned application but despite being quite numeracy literate I struggle with the statistical concepts and analysis of data.  I only received 65% for the first of five assignments.  Today I battled with the concepts of reliability and validity with all their different coefficient measurements.  The concepts in themselves are fine but when I have to apply them to a technical academic article it becomes all muddled up and difficult to navigate.  Not only do I need to understand these concepts for EDCN800 but I am also writing a literature review for EDCN806 which requires an examination of the reliability and validity of the articles I am including in the review.  It is all driving me insane and I question my ambition to complete a PhD down the track.  As a result I’m feeling a fair bit of empathy with my students at the moment.

    Anyway, that’s enough complaining, I need to attack a question about evaluations using numerical ratings and then write some of my own questionnaire items to assess student experience in studying the Masters of Education.  It is so much easier to help my students design their research for their Personal Interest Projects (PIPs) in HSC Society and Culture than to do it myself at a university level.  But here I go…

    Stay tuned.


Skip to toolbar