Evaluating EdTech: A Strategy for Selecting Digital Tools

Type: Article
Topics: School Administrator Magazine, Technology & AI

March 01, 2017

Students around tablet taking notes
Students taking notes around a tablet.

Bewildering. Confusing. Overwhelming. Those are some of the milder terms Ive heard educators use to describe the landscape for educational technologies. The good news: If you have a learning requirement in your school district, theres probably a mobile app that promises to solve that for you. The bad news: There are likely to be hundreds, if not thousands, to choose from. On Apples App Store alone, more than 80,000 learning apps are available.

Navigating through EdTech options can feel like a tightrope walk for school administrators hoping to balance the desire for innovation with the need to make fiscally sound choices. Fortunately, it does not need to be a binary choice. Over many conversations Ive had in recent months with digitally savvy educational leaders, Ive learned that its possible to do both.

Id like to share with you what Ive learned.

Where to Start?

Most educators Ive spoken with agree that defining your need is a good way to begin.

Lets say you want kids to learn more math, says Steven Hodas, former executive director of the New York City Department of Educations Office of Innovation and now practitioner in residence at the Center on Reinventing Public Education at the University of Washington. An administrator might see the problem as kids doing poorly in math. But a teacher might say that the problem is theyre unable to give students sufficient individual attention. Both point to kids learning more math, but they define the nature of their objectives differently.

Hodas suggests administrators engage people who work with the problem on a daily basis to help refine the need before jumping to possible solutions. That also includes parents, school staff, classroom aides and even the community at large. Once the need is better defined, it becomes easier to narrow down the universe of digital tools to the ones that address your need.

Next, Barton Dassinger, principal of the Ces獺r Chavez Multicultural Academic Center in Chicago, likes to proceed with a research question turning the need into a testable hypothesis. This forces people to think of ways to quantify their objectives. An example of a research question is: Do students using product X show higher gains in Y than similar students not using the product? where Y is the metric used to gauge progress, whether thats behavioral referrals, attendance, college admissions or test scores, Dassinger explains.

Before you make a purchase, you can work with the company to provide the data that will help you answer your research question, adds Dassinger, who recommends pressing vendors for weekly data reports. That way, you can evaluate their claims on the metrics that matter to you.

Improving Discovery

With tens of thousands of choices, finding the right tool can seem like searching for a needle in a haystack. Many educators have devised their own systems for filtering in the most relevant tools while remaining open to changing possibilities.

The Omaha Public Schools in Nebraska developed an app approval process that allows any educator in the school district to submit recommendations for digital products throughout the year. Since the process launched last fall, Omahas teachers have submitted more than 200 apps. Most receive a decision within five to seven days. If the app is rejected, the teacher receives feedback on why the product didnt pass muster.

Before this system, it was a manual process that would take months to process a recommendation, says Rob Dickson, Omahas executive director for information management services.

In Chicago, Dassinger prefers to interact directly with teachers within the classroom environment and have informal conversations to surface new products, keeping a list of prospects on a spreadsheet.

Shared Characteristics
 
Jessica Lindl laughing with students during a visit
Jessica Lindl, general manager of Common Sense Education, during a classroom visit.

These examples suggest theres room for a wide range of approaches. But the most effective systems tend to have several characteristics in common they are ongoing, transparent and adaptable.

  • Ongoing. Technological advancements occur nonstop, and teachers needs dont always coincide with adoption cycles. The best systems continually evaluate whats new in the market.
  • Transparent. Danny Wagner, manager of Common Sense Educations STEM Education Content and a former science teacher in Boone County, Ky., believes teachers need feedback so they can better understand why their submissions did or didnt make the grade. This is critical. Feedback shows teachers their voices are valued, and it will help them stay involved, providing a fresh flow of ideas.
  • Adaptable. Dickson calls his system in Omaha a workflow for change because he can update it with new curriculum requirements and learning standards. If we create a sound system for evaluating apps, it shouldnt matter what new technologies are thrown at us, he says. We can still do whats best for our kids.

If your districts discovery process involves issuing a request for proposals, Hodas recommends focusing on the learning goal, not the solution. If your RFP contains a list of specifications, people try to check the boxes and tell you what you want to hear. Thats not likely to be productive, he advises. You need to reach out in a way that gives people the freedom to give you creative solutions.

Filtering Options

Lets assume your discovery phase leads to a flood of options. An efficient way to narrow the field, Hodas says, is to ask teachers to watch a five-minute demo of each product (most tech products have video demos posted on their websites). After viewing, ask the teachers to answer this question: Would you use it in your classroom? If the answer is no (or mostly no), toss it out of consideration.

Once you have a manageable list of candidates, you can put them through a more rigorous evaluation. In Omaha, reviewers use a wide range of criteria, from compliance with the federal Childrens Online Privacy Protection Act and best practices to pedagogy and alignment to P21s Framework for 21st Century Learning.

In the Charlotte-Mecklenburg School District in North Carolina, Valerie Truesdale, the chief technology officer, says she first determines whether the products are interactive and adaptive. Students have to be interacting with the software and not just be given digitized worksheets, she says. And the software has to adjust its pace to individual progress so every student can have a personalized learning experience. Products then are evaluated using North Carolinas Standard Course of Study.

At Common Sense Education, where weve reviewed more than 2,500 educational technologies, our rubric includes engagement (how compelling the product is), pedagogy (how well it supports deep learning) and support (how accessible the experience is to students and teachers).

Recently, we added a fourth pillar: privacy. Over the past two years, weve worked with more than 100 school districts to create two free resources for schools: (1) a program that evaluates educational software products on their safety, security and privacy policies, along with how well they comply with government regulations; and (2) an open source toolkit for districts that want to do their own evaluations. By the end of the school year, our director of privacy review, Bill Fitzgerald, plans to have completed evaluations for 300 educational technology products on our site .

Start Modestly

Finally, once youve selected a technology to try out, start small and iterate.

Think of the first implementation as a pilot, Hodas advises. Dont think of pilots as free software to try out. Think of the vendor as someone who is collaborating with you to create a product that will result in the changes you both want to see.

This can be a cultural leap for school districts used to being given fully developed materials from established publishers. With technology companies, particularly the younger and more innovative startups, products are in constant evolution, and their developers are continually adding features, fixing bugs and improving the overall experience. While it can be frustrating to deal with a moving target, schools can take advantage of that flux by jumping in to influence which features get added and how the product is improved.

At the end of the day, I am reminded of some sage advice for maintaining perspective, provided by Truesdale, a veteran school leader in the Charlotte-Mecklenburg Schools. The key is to remember that these are all digital tools, she says. Just like building a house, you have many tools in your toolbox. What tools we use depends on the job we want to accomplish. But theres no substitute for quality teaching. 

Additional Resources

Common Sense Education, a nonprofit organization, provides educators with free, research-based classroom tools to help schools harness technology for learning. The website  includes a primer for evaluating educational software with particular attention to child privacy, teaching strategies and practical ideas on digital citizenship.

Also, Common Sense Education suggests these information sites:

  • ConnectEd, 
  • Digital Promise, 
  • EdSearch/Lumos Learning, 
  • EdSurge, 
  • Federal Trade Commissions explanation of the Childrens Online Privacy Protection Act for educators, 
  • LEAP Innovations, 
  • Learning Innovation Hub, 
  • P21s Framework for 21st Century Learning, 
  • Technology for Education Consortium, 
  • U.S. Department of Education, Office of Educational Technology, 
Questions To Cut Through the Marketing Spiel

Whether youre putting out a formal request for proposal or you just want a conversation with the vendor, here are some questions to help educators move past the marketing script and on to how the proprietors products can improve student learning. Ive collected some of these from Danny Wagner, head of Common Sense Educations STEM Education Content, to spark your thinking.

Customization: Can the company customize the product to fit the needs of a wide range of schools and populations in your district? Can teachers create custom content? How well does the tool integrate with other tools teachers are using in your classrooms?

Connectedness: How does the technology help connect stakeholders, including parents, students, teachers, administrators and community members? The stronger these connections, the more students will take ownership of their learning, teachers will become engaged facilitators, parents will become decision makers and empower learning at home, and businesses will give their time, sponsorship and donations to educational initiatives.

Equity: How can a variety of student populations access the product? Is it available in multiple languages? Can resources be printed or read out loud? Are there options for library access or home access?

Technology Requirements: What are the platform and systems requirements? Does it work with existing school devices? Does it require the school to upgrade its infrastructure? What support does the company offer schools to help integrate the product? Can the product or service scale to support thousands of students?

These will surely prompt a dozen other questions for school district leaders to ask. If so, you are well on your way to being an expert educational technology evaluator.

Advertisement

Advertisement


Advertisement

Advertisement