EdTech Products & Analytics⌗

Some time last year, I read a research paper called Dashboard stories: How narratives told by predictive analytics reconfigure roles, risk, and sociality in education (s/o to Juliane Jarke & Felicitas Macgilchrist as they are the original researchers and authors of the paper). I wrote a whole end of course paper on the research so I could go into a lot more detail, but I’ll keep it relatively high-level on this for the sake of all our attention spans. The purpose of the research was to gauge the impact of predictive analytics on how teachers make decisions (framed through a focus on data dashboards with ML available to them through a variety of edtech products). This paper has haunted me a little bit over the last couple months as I’ve spent time building products and seeing other products be built in the edtech space and I felt like venting into the void.

The main thesis is that there are specific narratives & stories that are “written” by predictive analytics and consequently, into the dashboards that are produced, which then alter the behavior of teachers and students. These dashboards and products shift the teacher role to resemble a manager, which in turn, shift the framing of a student’s actions & potential to that of interactions that are “machine-readable”. To me, this seems related to the overall trend of “Silicon Valley best practices” spreading everywhere and the collective obsession modern business seems to have with FAANG culture & “data science” (note: I love a good dashboard as much as the next guy, but not everything needs to be tracked and fed into a model i.e. the educational system and the complex graph of interactions that make up its foundation). One could argue that this trend has even reached the other side of our educational system, parents and the home (see The Atlantic Piece with the all time quote, “Perhaps one’s children and direct reports are not so different after all."). I guess my general take on this phenomenon is: data is not neutral, so the implementation of data analytics into an aspect of society is not inherently a good thing, and doing so blindly is such a giant cop-out of agency and a grand example of indeterminate optimism (“ML/data science/analytics dashboard is the future of education” & techno yoga-babble instead of idk actual investment and interest in building out our country’s educational system at the local and federal level being the future.)

Anyway, the paper goes on to expand on dashboards as storytelling devices with two main narratives taking place. The first narrative deals with risk. If we conclude that predictive analytics and the dashboards shift the teacher into a “manager” role, then it stands to reason that as managers, they are being pointed towards seeing risk as a key dimension in the classroom and their interactions with students. Anyone that has managed a project of any kind knows that risk is something that has to be considered and measured, so this pivot in the role of teachers makes sense intuitively. Thing is, risk dashboards only show correlation, but the narrative nature of how humans interpret the data (humans love stories) shown in dashboards emphasizes causal connections, which leads to faulty conclusions that correlation is the same as causation from a behavioral perspective (big no no). The other narrative is about sociality. Sociality is a dimension that can be measured in predictive analytics but unfortunately, the nature of data at this point in time means that sociality can only be measured from “in-tech” interactions. This means that the only data points used to analyze sociality in students come from digital communication (discussion boards, comments, etc). Once again, anyone that has been assigned a discussion board knows that being measured solely on a db post as your “sociality” dimension is not a good representation of your true sociality (“Good evening xxx, Great post! I definitely agree with blah blah blah” at 11:55 PM before your initial post is due). This constraint leads to interactions in the physical classroom to be under measured when drawing conclusions about student behavior and participation. The conclusion of the paper finds that the constraints involved in predictive analytics and dashboards in the education space can lead to a “narrow understanding of individually measurable education” (shocker) and can render the external, structural inequalities invisible and thus not considered when teachers try to turn dashboard insights into action. In english….not all of education or life is digital, and measuring those complex aspects of life & society through the lens of only the digital interactions & inputs that can be modeled through software is extremely limiting and big NOT GOOD.

With all that in mind, let’s turn to the world of tech conferences. Recently, I attended a certain edtech conference that was A) gigantic and really cool but also B) full of products and vendors with DASHBOARDS & CLASSROOM ANALYTICS & MACHINE LEARNING and it just really struck me how much we’ve deferred to Tableau and D3 and python and all the buzzwords in the world instead of taking risks and exploring new ways to create technology for the actual purpose of education (not tracking, but like you know passing on knowledge and preparing the future of our society). As tech has risen as the Industry and its quirks and sins and general THOUGHT LEADERSHIP have spread, I fear we’ve forgotten what technology can be: a catalyst for rapid change and transfomation that has yet to be matched. We don’t need programs that track users across browsers or log keystrokes or any of the surveillance mechanisms we seem to implant into every product and feature list. We need products that can reach more students at a higher quality both in-person & remotely, expand access to actual broadband & high speed internet, add to our teachers' natural abilities, and enrich students’s experiences, not just track // document them. As a builder of these things, I can only do my bit to create products that respect the balance between the analog and digital components that make up a healthy life but also I can have my scream into the void, which is this piece. Please stop tracking keystrokes and start trying to teach.