Below is an example of a fact-check draft I wrote recently. It’s a bit shorter than what I’m asking you to do for your final Truth-o-meter post (~800 words), but I hope it gives you a sense of what I expect. Note that Caulfield’s strategies are in bold.
I decided to fact check a recent news item entitled,”Study: Pennsylvania, New Jersey Rank Top 5 Best States For Teachers” from local broadcaster CBS Philly. I thought the headline was relevant to many of us since I’m a teacher and many students in WRT are planning careers in education. I came across this specific article by searching “recent study” in Google News. This wasn’t the first hit in my search, but it was up there, presumably because Google’s algorithm knew I was searching from Collingswood, NJ and thought I would find this piece relevant and meaningful.
Strangely, when I clicked on the source, I was taken to CBS Philly only to see a video at the top of the story about a teacher’s strike ending in the PA district of Methacton.
Yet, when I read the text below, I saw the following lead:
When it comes to teaching, it doesn’t get too much better than Pennsylvania and New Jersey, according to a recent WalletHub study.
As I read more, I saw that the study found that New York was the best state to teach, followed by New Jersey. This was good news considering I was educated in NY most of my life and now teach here. Although the news story didn’t tell me if by “teacher” this included post-secondary instructors such as myself, it did share a little about the study’s methodology. Wanting more, I went upstream by clicking the link to the suspiciously-named publication cited in the news item, WalletHub, which advertises itself as a service for free credit scores and reports. This was not encouraging.
Before I dove into the study, I used Google to read laterally to find out a little more about WalletHub. According to Wikipedia, “WalletHub is owned by Evolution Finance, Inc. – parent company of the credit card website CardHub.com.” One of the citations takes me to the Wall Street Journal, which I cannot read because of a paywall, but a 2014 headline doesn’t inspire any more confidence: “Credit-Card Comparison Sites Come Under Fire.”
Still, I also learn from Wikipedia that although WalletHub monitors credit, it also regularly pumps out clickbait-friendly, listicle-like reports such as “2017’s Most & Least Diverse States in America” and “2017’s Best & Worst States for Women.” The study from the CBS Philly story is linked and so I can use some of the tactics from Caulfield’s Web Literacy for Student Fact-Checkers to see how legit it is. I already know, however, that it’s not an academic journal and there aren’t any authors listed. This certainly limits its credibility as a scholarly source, but when I search for Wallethub on Google Scholar, I see they’ve actually been cited in subsequent studies. Then, as a I skim the content of this study, I see that they interview 11 professors and experts from various centers and universities, asking them questions like “What are the biggest issues teachers face today?” This is meant to provide better context, though the study doesn’t say how it incorporated these experts’ responses into the methodology for the study.
Still, the methodology is explicit, arguing that two factors weighed most in their rankings. On a 100-point scale, the researches weighted “Opportunity & Competition” as 70 points and “Academic & Work Environment” as 30. The former included stats on average salaries, long-term growth, pensions, and tenure while the latter ranked based on the quality of schools, pupil-teacher ratio, union strength, and annual credentialing requirements.
In all, while WalletHub’s study might still be useful, especially for preservice teachers who are considering where they want to live and work after college, I would have to analyze the 11 interviews to arrive at some sense of consensus on the criteria for measuring the best and worst states for teachers (and even then we are talking about only 11 people, not the entire field of educators). Since the study didn’t remark on this data, I’m not sure if their criteria is really all that solid. In just browsing the interviews, I see that the factors that retain teachers are complex and not so easily converted into a methodology. Moreover, while there are some citation for the data at the bottom of the post, I have no idea which information noted in the study came from which sources. This is a problem.
As Caulfield mentions, we should also know “who would be the most trustworthy source” so that we don’t always let sources lead us to where we go, but question who is most authoritative on the topic at hand. There are other places I could go, then, if I was curious about which factors make teachers fulfilled and happy at their jobs. I could fact-check (painstakingly I think) some of the the data on salaries, student-to-teacher ratios, etc., alluded to in the study. But I could search for keywords like “teacher happiness” in Google Scholar to see who researches this topic in great detail. There was also this study, that I found from the National Center for Education Statistics (NCES). In all, I could go deeper to try to develop a sense of consensus on what makes a good setting for teachers, but WalletHub, surprisingly, isn’t as terrible of a place to start as I originally thought.