SU’s Student Online Self-Development programme: a development opportunity or a listening exercise?

Self-development courses are an important aspect of personal and professional growth – but how do we determine their quality and to tell if certain opportunities serve a different purpose?

The Idea

When I first saw the advert for the programme, I was excited; Santander Universities UK and Sporting Edge – a (sports) consultancy providing psychoogy-based coaching and training programmes – teamed up to provide a self-development programme for university students. Students of Santander partner universities were able register for the programme – the spaces were limited – and the best four had a chance to win a £10,000 Santander Development Grant. As a psychology student, I was naturally eager to take part for two reasons – to (a) make the most out of the programme and (b) to see how they go about it. Of course that winning the would have been great, too!

Long story short – I was disappointed. The content was not bad at all – the programme provided very interesting insights from experts in different areas – but the programme coherence, overall delivery, and assessment could use much improvement.

The Programme

The Programme is delivered through the Sporting Edge’s Performance Zone – an online learning platform quite similar to those of MOOC (i.e. Massive open online courses) providers (e.g. Coursera or Future Learn).

The programme is completely asynchronous (i.e. all the videos are pre-recorded) and passive (i.e. there are no opportunities to reflect on the content, discuss your ideas, or share your experiences with other participants and/or course creators).

Each unit (or Topic) consists of 3 sets of four short insights (i.e. videos lasting 0:50 – 2:30 minutes), a short summary of the content (‘Key messages’), ideas for development/reflection (‘Next actions’), and further resources (articles, videos, books).

There is a quiz at the end of each of the units that consists of 12 multiple-choice questions.

(click for the programme overview page)

The Delivery

The delivery of the programme is one of its weakest points. Let me explain why.

Lack of directionality. The programme provides a lot of interesting content, but the way the content is presented is just awful. There is no solid introduction to the course, no content overview, and no learning outcomes. The topics are presented with video thumbnails and very brief summaries of their content. which however do not provide a sense of ‘going somewhere’ and at times, working through the content feels like watching a random video playlist. A set of short videos introducing the whole programme, individual topics, insights, and summarising the ideas discussed in each topics would have been wonderful as well as a list of learning objectives and takeaway points that could be printed and used to tick objectives off.

Lack of coherence. In my opinion, the insights are not well linked – I did not experience the logical sense of transitioning to a related when moving though the learning units. The sense of ‘moving somewhere’ was not there for me. Let’s take the first topic – Personal Drive – as a n example; it covers setting dream goals, turning dreams into a reality, and understanding own motivation. We could say that there is a relationship between these topics, but what is it like? How does one relate to the other? This is something that should be clearly outlined. In other words, if this project were my essay, I would get marked down for a lack of coherence.

Lack of active participation. The programme focuses on self-development. Okay, fair enough, is self-development something that happens in isolation? Hardly. The sole point of this programme is to share experts’ insights and experiences so that we can learn from them. Why not let the participants share their insights and experiences? Psychology tells us that peer learning (i.e. learning with/from individuals on the same level as ourselves) is one of the most powerful ways to learn (here is some evidence – browse and have a read). The next action sections have some great questions and reflection points – this is, in fact, the only active learning element as it requires us to actively think about something. Why not make use of discussion boards or chat rooms so that the participants can share their views? Most MOOC providers recognise the importance of peer learning – it is, in fact, an important part of many of their courses.

The Content

The content is the strong point of the course. The pre-recorded videos contain interviews with experts in different fields (e.g., psychology and management) and athletes. Both groups make valuable contributions – the experts explain concepts (e.g. the fixed and growth mindsets) and the athletes share their experiences. This is a nice way to link theory and practice. If more attention is given to clearly linking the two phenomena, the programme could make quite a few very strong points.

The takeaway message for each of the insights is nicely summarised in the Key messages section. This is very useful for consolidating the newly acquired information. The key messages are quite general as the videos are mostly based on individual experiences and can be serve as a content overview that can help participants focus on the key ideas.

Next actions, the final section of the content, is an important section (likely) intended to generate active thinking of one’s state of affairs, goals, motivation, and promote self-reflection. This is a great idea! However, as discussed above, the participants do not have a chance to share their insights, discuss their ideas, or learn from others. If this were the case, the section would likely become the most important learning tool of the whole programme.

An example of the sections. Click to enlarge.

The Extras

The extras, labelled as ‘Want to learn more,’ are my personal favourite. I have thoroughly enjoyed watching extra videos, reading external articles, and I even ordered a book recommended on the course. It’s called Mindset: Changing The Way You think To Fulfil Your Potential and it’s brilliant. I think that everyone interested in personal and professional growth should read this.

Apart from articles, videos, and literature, this section offers useful tips that everyone can implement. You can, for example, learn how to schedule time for your goals using Google Calendar. Very handy.

If I’m honest, I think that the extras alone would be enough to provide enough content for another course. This is definitely a strong point of the course.

The Grant & The Quiz

I’m sure that the £10,000 Santander Development Grant was reason enough to participate in the programme for some of the participants. It was awarded to 4 individual with the highest quiz scores – but there’s a twist, of course.

The Assessment

There were 96 (8×12) questions altogether. The highest scoring individuals were in for quite a treat – £10,000. I spent quite some time preparing for the first quiz – I was taking notes, highlighting important information, reading through the extras, and using the Next actions sections to reflect on my own situation as I was expecting sneaky, abstract, and philosophical questions focusing on the understanding of the topics covered – but boy was I wrong. This is one of most frustrating and ridiculous parts of the programme – and all the more so when one realises that the platform is run by a psychologist who, in theory, should be familiar with the principles of testing and assessment.

I study psychology. I am also a qualified EFL (English as a Foreign Language) teacher. Drawing on my experiences from both of the fields, I knew there was something wrong with the quiz the moment I opened it. Let’s pick one quiz question to illustrate the problem.

In his video, Graeme talks about goal-setting. He is using the example of placing a reminder of his dream goal [to play cricket for South Africa] on his fridge when he was 11 to (arguably) explain that writing down his goals was something that ‘came from’ him and that it helped him to formulate and follow his goals.

An example of a quiz question. Click to enlarge.

A self-development programme is not a listening exercise. When learning EFL, we focus on (a) getting the gist (i.e. general information) and (b) extracting details. This is what people normally do when they listen, but then you are learning a second language, it is good to point out these two strategies as they both require a different approach to listening. I was quite shocked when I realised that some of the quiz questions very closely resemble Cambridge English EFL listening papers I was using to prepare elementary and intermediate students for their exams.

Why is this a problem? Well, the learners’ task was to simply extract a piece of information from the listening exercise, they were not asked to think about the underlying principles, the context, or the takeaway message. Higher-level listening papers still want you to extract information, but they include distractors (i.e. information that superficially resembles the topic in question) to make sure you really understand to what you are listening to when they are selecting the answer. Learners sometimes experience difficulties when faced with this type of task – mostly because they start thinking about the ‘deeper’ meaning too much instead of focusing on the information only. Do you see the issue now? No? Okay, check this out.

Testing understanding is concept testing, not content testing. This issue is closely related to what we’ve just discovered. We know from psychology that when we want to test a latent trait (i.e. something hidden that’s not directly observable), we have to devise a means of testing it indirectly. To put it simply, understanding and knowledge are not phenomena we could directly see or touch. We have to come up with good testing strategies. A quiz is an example.

If you read the video description, you will (most likely) conclude that Graeme’s strategy to formulate and follow his goals was externalising them – writing them down and putting them somewhere visible. That is arguably the (latent/hidden) concept (i.e aiding one’s goal formulation and following by creating an external reminder) illustrated by the video content (i.e. Graeme talking about sticking his goals on the fridge) – this illustrates very nicely that what is said and what is meant is not necessarily the same (if that is unclear, imagine your sibling rolling their eyes and saying “That’s just great!” when you tell them, amidst their [insert a suitable game, e.g. Fortnite] match, that you need the computer because you have homework to do – then compare the actual utterance with the latent message).

Now back to Graeme and his magical fridge – does the question test the content knowledge? Yes.
Does it test our understanding of the concept? No.
Is this okay? No!
Why? Because we don’t need to understand the concept when we’re answering content-based questions. And that is the major issue of this quiz and assessment in general. It’s not easy to make a good test!

So, how could we test the concept rather than the content in this case? Let’s try!

Q: What is a good strategy to formulate and follow our goals? (Hint: Think about Graeme’s story *if you want to make it explicit*)

  1. Make them visible so that you can see them frequently (write them down, draw them, etc.).
  2. Repeat them before you go to sleep every night.
  3. Make sure to tell everyone so that they don’t distract you from achieving them.
  4. Don’t tell anyone – goals are like wishes, you mustn’t say them aloud!

See? It’s not that hard. If you’re after the meaning, not just the words, you will be able to grasp the concept.

You can wing it if you want – it’s not even hard to do. Well, if you’re after the grant only, surely there are some measures to stop you from cheating, right? No. I am aware that this post is long enough, so I’ll stick to bullet points:

  • There is not time limit to complete the quiz.
  • There is also nothing preventing you from opening the relevant insights in new tabs, which means that
  • you can pick a question, listen until you get to the relevant content, then record your answer and move on.
  • The only form of ‘protection’ is that you only have one attempt at each quiz – having said that, using the abovementioned strategies completely eliminates its function.

To summarise, the quiz is just messed up.

The Terms

Alright – even though the testing is flawed, it is still logical to award the grant to those who score the most points, right? Right. But what if we get more than four individuals with equal top scores? Well, let’s come up with something we can easily measure that will help us eliminate the others.

This is an excerpt from the T&Cs. Click to enlarge.

The Terms and Conditions have further criteria in place to decide who is going to win in case there is a tie among the quiz scores. Now there are two ways(one slightly better than the other) to interpret this as the T&Cs are a bit vague:

  • Participants who have less views programme-specific insights and additional insights will get the grant. This would be a disaster and I hope this was not the case because it would mean that people who revisited the videos would be punished for doing so. It makes sense to go back to what you’ve previously studied – that’s how we consolidate knowledge.
  • Participants who have more views of programme-specific insights and additional insights will get the grant. This is slightly better than the first scenario as it reflects the fact that those who are interested in the topic would revisit the insights, but it has a major flaw – those who scored 100% and also happened to have looked into the T&Cs may have organised a clickfest to inflate their views. Again, not helpful.

The Conclusion (tl;dr)

Well. As you can see, there are quite a few issues with this programme. Even though the content is good and interesting, the summaries useful, the reflection points very good and the extras amazing, the coherence and the delivery of the content is lagging behind dramatically.

The quizzes are just plain bad. They do not look at the underlying concepts and ideas (which are the takeaway points in the end) as they only test what you hear in the videos.

Could Santander Universities and Sporting Edge have done any better? Well, yes. They could have followed the format of other MOOCs – quite a few are free to enroll onto.

What about the quiz? They could have thought of the questions more. Or scrapped the questions altogether and use essays or portfolios instead. Those are the tools that show how dedicated someone is to their learning and development. And they also take a while to assess. And time is money. But hey, so is the £40,000 awarded through the grants.

Before you close this article and move on with your life, I’d like you to consider this: Are the individuals who received the grant those who worked the hardest, or are they the people who know how to play the system?

…I guess we’ll never know!

Disclaimer: No copyright infringement intended. The images are, unless stated otherwise, property of Sporting Edge Solutions Ltd. Should the copyright owners with for the images to be removed, they are kindly asked to contact the website administrator who will remove them immediately.

Published
Categorised as Blog, Reviews

By Ondrej Hoberla

Live & learn.