Despite the promised potential of adaptive learning systems, many questions remain unanswered. There is not yet enough research or documentation of classroom practices that help broach these issues:<\/p>\n\n
When developing ALS, some principles are used either directly or implicitly. These are not always without consequences.<\/p>\n\n
What\u00a0do machine learning systems do when they predict or recommend something? They use the student\u2019s past experiences, preferences and performance in order to choose what to recommend to them; they look to the past in order to predict the future. Thus, these systems are always biased towards the past4<\/sup>. Machine learning works best in a static and stable world where the past looks like the future5<\/sup><\/em>. ALS, based on machine learning models does more or less the same thing, but now with the addition of pedagogical considerations.<\/p>\n As a consequence, these systems are not able to account for fluctuations in normality like the COVID pandemic, health issues and other problems. They can struggle to take account of age, growth, mastery of new competencies and personal evolution of young humans.<\/p>\n Is student behaviour even predictable? How many times can we repeat a formula that worked well in the past, before it becomes boring, repetitive and impedes progress6<\/sup>? Even if such a prediction were possible, is it even prudent to expose students only to that which they like and are comfortable with? How much novelty is overwhelming and counterproductive6<\/sup>?<\/p>\nIt is difficult to decide how similar recommended activities should be, how many new types of activities should be introduced in one session and when would it be productive to push a student to face challenges and explore new interests. The answers do not lie in the students\u2019 pasts alone.\n Even where the past can be used reliably to predict the future, the past itself could be difficult to capture accurately. How can Youtube know a user liked a video? It is easier where they clicked the Like<\/em> button or subscribed to the parent channel after watching it. But such explicit behaviour is often rare. Recommendation systems have to regularly resort to implicit signals that may or may not fully reflect the truth4<\/sup>. For example, Youtube uses the time a user spent watching the video as an implicit signal that they liked the video and would like to watch similar content. But, just because a video played on someone\u2019s computer till the end hardly means the person liked it, or even watched it7<\/sup>.<\/p>\n What about how feedback is recorded in an adaptive learning system? To gauge, for example, if a student was attentive during an activity, the system might record the number of digital resources they clicked on, and when and for how long they accessed them. But these cannot accurately reflect their level of attention1<\/sup>.<\/p>\n For example, if the student is clear about what to do for an activity, they might consult a few resources and zero in on the critical points quickly. Someone who is not as clear might open and spend time on all the listed resources without learning much1<\/sup>. It is possible that the first student is wrongly flagged for lack of motivation and made to do additional work.<\/p>\n Also, the machine learning models can only note that two things happened \u2013 a student clicking on a resource and a student scoring high on the associated exercise. They cannot infer that the student scored high because they consulted the resource \u2013 they can infer correlation but not causation5<\/sup>.<\/p>\n The unfair expectation of some ALS is that the teacher will dive in and remedy such errors. In other systems, the teacher does not even have the option to do so.<\/p>\n\n Recommendation systems cannot handle multiple goals. The aim of the ALS is often put forth in a form of a single question: the surrogate question<\/em>. What rating did a user give a movie, how long did they watch a video, what is the score of the student in a quiz, how well did they satisfy the criteria the machine used to measure attentiveness\u2026 The systems are then trained to attain these goals and tested based on whether they was achieved. Their performance is constantly adjusted to maximize their score with respect to these goals.<\/p>\n If scoring on the quiz is the goal, certain content is recommended in a certain way. That way, exam performance is the surrogate problem that is solved. If the goal is just to make them click on many resources, recommendation would be tailored to push them to do just this. In this instance, making resources adequately attractive is the problem.<\/p>\n The\u00a0 choice of the surrogate question has an outsized importance on how the ALS works. What is more, contrary to the promotion of ALS as objective systems, there is more art than science in selecting the surrogate problem for recommendations4<\/sup>.<\/p>\n\n As we have seen, many decisions go into the making of ALS \u2013 what data is measured, how this data is used to gauge feedback and other information, what goals are optimised, and what algorithms are used to optimise these goals. Often it is programmers, data scientists, finance and marketing experts that are involved in making these decisions. The input of teachers and pedagogical experts in the development process is rare and often comes after the designing process2<\/sup>. Products are not field-tested before adoption in schools, and often their proclaimed effectiveness is based on testimonials and anecdotes, instead of scientific research2<\/sup>.<\/p>\n As a result, what a school needs and is familiar with has little impact on what software companies are building. Cost, availability and infrastructure can have a major say on what schools can buy. It is important to bear this in mind while deciding if or how to use a particular product. Perhaps it is better not to think of them all as adaptive learning systems or AI, but individual systems with wildly different objectives, designs and capabilities.<\/p>\n ALS as a whole can be used for personalising feedback, scaffolding and practice. They can find gaps in learning and remedy it within the limits of programming and design. They cannot detect \u2018teachable moments\u2019 or when it is right to capitalise on the mood of the class to introduce a new idea or example. These capabilities that make learning magical and which help the lesson endure in the student\u2019s mind are solely the forte of the teacher.<\/p>\n\n\nA paradigm of ALS: the explicit reflects the implicit<\/h3>\n
The paradigm of ALS: everything can be replaced by this one question<\/h3>\n
All tech is not hi-tech<\/h3>\n
\n