Hello World, As many of you have probably noticed, there is a growing problem on the internet when it comes to undisclosed bias in both amateur and professional reporting. While not every outlet can be like the C-SPAN, or Reuters, we also believe that it’s impossible to remove the human element from the news, especially when it concerns, well, humans.
To this end, we’ve created a media bias bot, which we hope will keep everyone informed about WHO, not just the WHAT of posted articles. This bot uses Media Bias/Fact Check to add a simple reply to show bias. We feel this is especially important with the US Election coming up. The bot will also provide links to Ground.News, as well, which we feel is a great source to determine the WHOLE coverage of a given article and/or topic.
As always feedback is welcome, as this is a active project which we really hope will benefit the community.
Thanks!
FHF / LemmyWorld Admin team 💖
You must log in or register to comment.
A lot of the criticism I’ve seen thus far falls into two categories:
- Users complaining that their favorite source is scored poorly
- Users complaining that the ratings have various sources of statistical bias
The ones in the first group I think should take it as a wakeup call that they are either headline shopping or missing out on other perspectives of current events. This is especially important on the international stage where armed conflicts will naturally produce two opposing accounts (and lots of propaganda).
The second group have a point - MBFC isn’t the end all be all, but it’s certainly better than nothing. Having meaningful bias measurements for each relevant scale would be impressive but way beyond what MBFC aims to do.
So all in all - I see this as a very positive change
Thanks! Your points are perfectly on target.
If we had any other api with parity of media bias / fact check, then we would have included it, but we only see paid, no api available.
But for now we have added a ground.news search link so that everyone can see a third opinion on it.
While I’m not as concerned with MBFC as many others are, why not use Wikipedia’s RSP as the datasource? Made by the most reliable user-generated platform in the world, it’s a great list of controversial sources and is completely open. Changes are also infrequent enough so that adding to the database by hand would be quite easy.
I also echo the concerns raised below on the uselessness at a glance due to the accordion hiding the only information and purpose the bot was created to serve.
Because the mods on this community would rather choose a source that they agree with than a source that’s reliable.
Mods, I appreciate this bot!
Deciphering media bias is tough, and finding 1 site that will ‘perfectly’ identify biases is an impossible task, but at the minimum having this bot show up on posts ‘gets people thinking’ about the credibility of their news sources.
MBFC doesn’t have to be the ultimate arbitrator either. If it is missing something about a specific article people can call it out in the comments. At the end of the day, the worst thing it does is add more data about a news source and I’m not gonna complain about that.
Actually, I’ve checked, and you’re an unreliable commenter. Sorry.
So is it time for a new news community then if the admins don’t want to listen?
You can even be better than that! You can make a community that fact check news article / news pages. Then we can add the threads from that community to the bot and have there news page specific discussion.
This will be great to have. Thanks!
Ground News Media Bias Fact Check Credibility: [**High**] (Click to view Full Report)
Ground News is rated with High Creditability by Media Bias Fact Check.
Bias: Least Biased
Factual Reporting: Mostly Factual
Country: Canada
Full Report: https://mediabiasfactcheck.com/ground-news/Check the bias and credibility of this article on Ground.News:
- https://ground.news/find?url=https%3A%2F%2Fground.news%2F%29%2CMedia Bias/Fact Check Media Bias Fact Check Credibility: [**High**] (Click to view Full Report)
Media Bias/Fact Check is rated with High Creditability by Media Bias Fact Check.
Bias: Least Biased
Factual Reporting: Very High
Country: United States of America
Full Report: https://mediabiasfactcheck.comCheck the bias and credibility of this article on Ground.News:
- https://ground.news/find?url=https%3A%2F%2Fmediabiasfactcheck.com%2F%29Media Bias/Fact Check Media Bias Fact Check Credibility: [**High**] (Click to view Full Report)
Media Bias/Fact Check is rated with High Creditability by Media Bias Fact Check.
Bias: Least Biased
Factual Reporting: Very High
Country: United States of America
Full Report: mediabiasfactcheck.comCheck the bias and credibility of this article on Ground.News:
- https://ground.news/find?url=https%3A%2F%2Fmediabiasfactcheck.com%2F%29
Media Bias Fact Check is a fact-checking website that rates the bias and credibility of news sources. They are known for their comprehensive and detailed reports.
Thanks to Media Bias Fact Check for their access to the API.
Please consider supporting them by donating.Beep boop. This action was performed automatically. If you dont like me then please block me.💔
If you have any questions or comments about me, you can make a post to LW Support lemmy community.I don’t like this bot but I love how blatant it is lol
Media Bias/Fact Check is rated with High Creditability by Media Bias Fact Check.
Good, I guess.
This is a god-tier shitpost.
Thanks for this Rooki!
removed by mod
This about about to spawn so many sidebar threads xD
It’s either going to be awesome, or hilarious. Probably both.
Any guesses for how long until the “we’ve disabled the bot for further testing and review” post? My bet is a month.
But seriously, this is pretty great. Thanks for putting that together.
That’s just introducing 2 more sources of bias
Both sides? Geeet outta here.
That’s literally what the other source being added called Groundnews attempts to do.
I understand your edgy take, but equivocating reliable and consistent mediators that accurately discern real news from propaganda with trash like Infowars as “more bias” is nonsense.
Yeah, I’m not saying all their work is worthless and I know they’re good enough for the most extreme sources of misinformation but to paint entire publications as not reliable based on the assessment of couple laypeople with an inherently narrow worldview (at least a very American-centric one) is the opposite of avoiding bias in my opinion.
Not entirely and unequivocally avoiding bias every time isn’t the “opposite of avoiding bias”, it’s an example of perfect being the enemy of good.
There may technically be inherent bias everywhere, but it’s at best useless and in practice harmful and inaccurate to lump MBFC in with grayzone and to equivocate in general.
Example from 2020:
“Biden is just another politician, like Trump”
Technically true that they are both politicians, but without recognizing the difference between Biden and trump, the states wouldn’t have student debt cancellations, no federal minority legal defenses, fifty plus liberally appointed judges, no reversal of the trans ban, no veteran health coverage for toxic exposure, no green new deal, no international climate accords, no healthcare expansion and so on.
or:
“who cares, it’s just another plant”, but arugula is a great salad green while a bite of foxglove can kill you.
It’s important to recognize the shades of grey and distinguish one from another.
How fucked is it that such a poorly written book has ruined the extremely useful phrase “shades of grey”?
https://mediabiasfactcheck.com/radio-free-asia/
This what scores you high credibility: “a less direct propaganda approach” for state sponsored media that is not critical of its sponsor
https://mediabiasfactcheck.com/al-jazeera/
And this is what scores you mixed credibility: “exhibits significant bias against Israel” for state sponsored media that is not critical of its sponsor (updated in Oct 2023 naturally)
Now every article published by Radio Free Asia is deemed more credible than those published by Al Jazeera despite the former literally being called a former propaganda arm of the state in their own assessment. Yes, good is not the enemy of perfect but this is clearly an ideological decision in both instances.
CNN also scores as Mostly Factual based on “due to two failed fact checks in the last five years” one being a single reporter’s statement and the other being about Greenland’s ice sheets. That doesn’t seem like a fair assessment to me
https://mediabiasfactcheck.com/left/cnn-bias/
So based on this I am supposed to conclude that Radio Free Asia is the most credible source out of the three at a glance.
removed by mod
“good is not the enemy of perfect”
Incorrect quote.
Your problem is making “perfect the enemy of good.”
You are
-
mistaking bias check for news reporting
-
making perfection the enemy of the good
-
arguing that mostly doesn’t mean mostly(spoiler, mostly does mean mostly)
-
“this is clearly an ideological decision”: No, your examples provided are both conclusions based on consistent objective standards, the opposite of ideology.
-
Yes, that everyone make a better picture. Instead of getting shouted at by one manipulative entity.
Have you looked into who runs Media Bias Fact Check? It’s pretty much as opaque as it gets for a website that claim to have an authoritative list of biases for hundreds of websites. Just because it’s a meta source does not make it any more credible than any other random website.
Have you looked into who runs Media Bias Fact Check? It’s pretty much as opaque as it getsI haven’t even tried to look for their about page or an FAQ.ftfy
Not quite as opaque as it gets, certainly.
Media Bias Fact Check, LLC is a Limited Liability Company owned solely by Dave Van Zandt. He also makes all final editing and publishing decisions.
Yeah, looks great to me.
That’s a fair criticism. It is not opacity, however. The full real name of your lead guy is transparency.
How do you verify who these people are? For all you know it’s just a bunch of fake names on a page.
That’s true of all names. At a certain point you can simply decide to trust nothing if that’s what you want. Plenty of people do, though personally I think that’s foolish due to the pointless nihilism it results in.
Have you ever investigated every news page for its bias? With no pay? I guess not. In the end there is a human doing that manually.
Because of that we added the ground.news search url, so that if you didnt believed it you can get other news pages thoughts on this article.
Uh, you know that the information is right there, right? It even says where their sources of funding are: ads that are based on your browser history (e.g., shit like AdSense), individual donations, and individual memberships.
I’m not talking about their source of funding but their qualifications in making claims with such broad implications. It looks like the pet project of some guy and couple faceless names who do not even claim any meaningful professional or academic experience.
Here’s an example from your link:
Jim resides in Shreveport, Louisiana with his two boys and is currently working toward pursuing a degree in Psychology/Addiction. Jim is a registered independent voter that tends to lean conservative on most issues.
MBFC is entirely the opinion of some guy and his team of mystery helpers.
It’s pure garbage and one look into it shows how pathetic the biases are.
I love this, but I would like to state that Media Bias Fact Check seems to have a pro-Israel bias.
https://mediabiasfactcheck.com/mondoweiss/
- Overall, we rate Mondoweiss as Left Biased and Questionable due to the blending of opinion with news, the promotion of pro-Palestinian and anti-zionist propaganda, occasional reliance on poor sources, and hate group designation by third-party pro-Israel advocates.
I feel like “blending of opinion with news” and “occasional reliance on poor sources” is all that really need be said.
Mondoweiss is literal propaganda tho
It’s about the bias rating. Using explicitly biased sources when rating a source makes for a bad rating.
On whose behalf? I’ve sensed bias from the brief glances I’ve given them, so I didn’t keep reading enough to actually analyze it.
Read the lengthy discussion about them on wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Noticeboard/Archive_427#What_is_the_reliability_of_Mondoweiss?
We don’t allow Mondoweiss links either.
I independently checked Mondoweiss using Media Bias a few months ago because it was posted elsewhere and I had not heard of it before, but was disturbed to see the extra reasoning behind the rating.
It’s for sure questionable at best, the Wikipedia discussion someone else posted was enlightening on that, but “designation as a hate-group by pro-Israel” sources doesn’t really mean much when sources like the ADL equivocate anti-Zionism and anti-Semitic rhetoric in bad faith.
https://www.washingtonpost.com/nation/2024/06/26/wikipedia-adl-jew-zionism-israel/
Again, I love the bot, but wanted to state something to be conscious of
Yeah, I looked into it as well, there was someone intent on repeatedly posting Mondoweiss links and they would always get reported.
It’s trash tier reporting.
could you have the bot automatically unvote its posts (make it 0) so it goes under new comments when sorted by votes?
the spoiler thing doesn’t work on eternity and it kinda hides everything under it being so long
I wish bot comments didn’t count toward the comment count, too. It’s annoying to see “1 comment” and then you look and it’s just this or the summary bot.
While I love the idea, I KNOW that there are certain groups that will refuse to accept that factual information. Tankies, for instance, will refuse to accept any criticism of their preferred sources. (As will Russian-asset Jimmy Dore.) Far-right conservatives will do the same, only on the other end of the spectrum.
MBFC is not factual.
It’s subjective. The opinion of one random man on the internet and his supposed volunteers.
I’ve seen it rate Indian papers low and add comments like “Never once reported anything false.” Meanwhile some US garbage will be ranked as reliable and the comments are an essay on all the times they’ve been busted lying.
Got an example of a US source being rated reliable despite failed fact checks? I’d be interested in seeing that.
I have another one - MBFC rates a site called UNWatch as “highly credible” when in fact they run trash-tier hit pieces on UN officials who criticize Israel. Their articles have been removed from [email protected] for disinformation.
I debunked one of their articles last month. If you want to see the kind of crap they publish, see a screenshot of my critique here.
Yeah that’s concerning. You could send the MBFC people an email with your evidence and see if they can take another look at the source, that’s a pretty niche one they probably won’t re-check very often unless someone requests it.
Someone up-thread posted an MSN story about Hamas killing babies, and MSN’s high rating. As we now know, that story was an Israeli fabrication.
That won’t quite qualify.
Dozens of babies were brutally murdered — some even decapitated — by Hamas terrorists inside a kibbutz in southern Israel during Saturday’s shocking assault on Israeli civilians, according to journalists who were let in to see the aftermath of the massacre and corroborated by the Israeli Defense Forces
The key is that according to journalists part. If the sources lied to the outlet, then the sources lied. This is not the fault of the outlet, and does not mean they shouldn’t have reported it. That said, that probably does deserve a retraction.
To fail a fact check, you have to publish something known at the time to be misleading. Otherwise it’s a mistake, and should just be corrected when more accurate information arises.
I am disappointed that the article has not been corrected by now, however.
They failed an Al Jazeera fact check because they published an article using data from the South African government that was later updated long after the article.
So yeah, it qualifies to this trash tier site.
The only fact check failure I can see with that SA article seems to be them citing their source as a UN report, when no UN source ever made any such report.
https://africacheck.org/fact-checks/reports/no-murder-rate-women-south-africa-hasnt-spiked-117
Otherwise you are right, and this would not qualify. You cannot cite the UN when the UN is not where you are getting your data though, that is blatant misrepresentation.
Great idea
I think having this post isn’t a great idea because you are just assuming the websites bias are legit. At the very least there needs to be a lot of warnings in the bots post about the websites biases and the methodology they use so the reader can come to their own conclusion.
Just looking over the methodlogy it’s clear that it has it’s own biases:
American Bias
The website itself says it’s distinctions of left and right are US based which is very skewed from the rest of the world. There should be a disclaimer or it shouldn’t be used in any world news communities.
Centrist Bias
The website follows the idea of “enlightened centrism” since if it determines a website has a left/right lean (again arbitrary) it affects the factual ratings of the sources.
Examples of this are: FAIR only getting the 2nd highest rating despite never having failed a fact check.
Despite my personal opinions on the pointlessness of using a US based left/right bias criteria I’d feel better if it was at least kept it it’s own section but when you allow it to affect the factual rating of the source it’s just outright wrong. The factual accuracy of the website should be the sole thing that affects this rating.
Questionable Fact Checking
Even just checking some of their ratings raises doubts on the websites credibility.
The ADL is rated as high (2nd highest) and wasn’t found to fail any fact checks.
“Wikipedia’s editors declared that the Anti-Defamation League cannot be trusted to give reliable information on the Israel-Palestine conflict, and they overwhelmingly said the ADL is an unreliable source on antisemitism.”
Maybe Wikipedia editors are a good arbiter of truth and maybe they aren’t but as people can see there isn’t a consensus and so by choosing Media Bias/Fact Check you’re explicitly choosing to align your “truth” with this websites biases.
This is a really well-reasoned response… Which probably means the mods will ignore it
I’ll add UN Watch to the list.
MBFC rates it as “highly credible” despite it publishing laughably bad hit-pieces on UN officials who openly criticize Israel.
I did a debunk on one of their articles that was removed from this very community due to disinformation, but I’ve posted a screenshot of my critique here for anyone who is interested.
A standard of factuality needs to include a provision of avoiding emotionally-loaded, manipulative language. Otherwise you can pump unlimited amounts of propaganda with full factuality simply by “asking questions”.
I wont disagree that there should be a ranking for using loaded language but combining it with the factuality ranking twists what the ranking means since to the average person they’re going to read that as how accurate the facts are.
It should be its own separate rating from factuality. Again if we’re going to have to have a bot like this put clear disclaimers and ideally find a better one than this.
I disagree. I think emotional language is fundamentally the opposite of real objectivity, and cannot be honestly acknowledged as factual in any confirmable way.
It has no place in objective discussions, and employing it in any way, shape or form makes one deserve objectivity demerits.
edit: And objectivity and factuality are synonyms.
This is one of the reasons why the right is so successful. By equating emotion with lies, they erase the objections of the oppressed, and can continue with a veneer of objectivity as they advocate for genocide by seeming non-emotive and rational.
Fact-based reporting should be a measure of whether the statements and facts, express and implied, line up with the truth, and nothing more.
The Intercept getting only a “mostly factual” rating (3rd highest) despite their admittance it has never failed a fact check.
This is literally in bold at the top of the page:
Overall, we rate The Intercept progressive Left Biased based on story selection that routinely favors the left. We also rate them as Mostly Factual in reporting rather than High due to previous fabricated work and censorship of writers.
Fabricated work.
Is there anything that’s more of a capital crime in journalism than fabricating quotes? Surely we can all agree that publishing fiction as news is the opposite of factual reporting? They may not have failed a fact check in the last five years but it just isn’t possible for them to have published fabricated news without ever failing at least one. By their own admission they failed five in that incident alone.
I’m not going to die on the intercept hill here I’m fine with the fact that even though they fired the person it’s a stain on their record so sure let’s say that rating is fine.
It was one of the first 3 I checked so I’m sure I’ll find more that are problematic when I have a chance to look because it’s their methodology that’s biased. Also the other 2 I pointed out are clearly not correct.
Got rebuttals for any of my criticisms about the methodology?
Got rebuttals for any of my criticisms about the methodology?
I do!
I think the importance of American bias is overstated. What matters is that they’re transparent about it. That bias also impacts the least important thing they track. People often fixate on that metric when it has little impact on other metrics or on the most important question for this community: ‘how likely is it that this source is telling the truth?’ Left and right are relative terms that change drastically over time and space. They even mean different things at local and national levels within the same country. It’s not really an MBFC problem, it’s a the-world-is-complicated problem that isn’t easily solved. And it’s not like they’re listing far-right publications as far-left. Complaints are almost always like, “this source is center not center-left!” It’s small problems in the murky middle that shouldn’t be surprising or unexpected.
It’s also capturing something that happens more at the extremes where publications have additional goals beyond news reporting. Ignoring Fox’s problem with facts/misinfo, it doesn’t really bother me that they’re penalized for wanting to both report the news and promote a right-wing agenda. Promoting an agenda and telling the truth are often in conflict (note Fox’s problem with facts/misinfo). CBC News, for example, probably should have a slightly higher score for having no agenda beyond news reporting.
It might matter more if it impacted the other metrics, but it doesn’t really. Based on MBFC’s methodology, it’s actually impossible for editorial bias alone to impact the credibility rating without having additional problems – you can lose a max 2 points for bias, but must lose 5 to be rated “medium credibility”. I don’t know why FAIR is rated highly factual (and I’d love for them to be a bit more transparent about it) but criticizing bias leading to them being rated both highly factual and highly credible feels like less than a death blow. If it’s a problem, it seems like a relatively small one.
MBFC also isn’t an outlier compared to other organizations. This study looked at 6 bias-monitoring organizations and found them basically in consensus across thousands of news sites. If they had a huge problem with bias, it’d show in that research.
On top of that, none of this impacts this community at all. It could be a problem if the standard here was ‘highest’ ratings exclusively, but it isn’t. And no one’s proposing that it should be. I post stories from the Guardian regularly without a problem and they’re rated mixed factual and medium credibility for failing a bunch of fact checks, mostly in op-ed (And I think the Guardian is a great, paywall-less paper that should fact check a bit better).
So I think the things you point out are well buffered by their methodology and by not using the site in a terrible, draconian way.
I think the importance of American bias is overstated. What matters is that they’re transparent about it. That bias also impacts the least important thing they track.
It affects the overall credibility rating of the source, how is that the least important thing? They also seem to let it affect the factual reporting rating despite not clearly stating that in the methodology.
Based on MBFC’s [methodology](https://mediabiasfactcheck.com/methodology/), it’s actually impossible for editorial bias alone to impact the credibility rating without having additional problems
This is only true specifically when you’re thinking about it as a great source can’t have its credibility rating lowered. A not great factual source can get a high credibility rating if it’s deemed centrist enough which again is arbitrary based on the (effectively) 1 guys personal opinion.
High Credibility Score Requirement: 6
Example 1
Factual Reporting Mixed: 1
No left/right bias: 3
Traffic High: 2
Example 2
Factual Reporting Mostly Factual: 2
No left/right bias: 3
Traffic Medium: 1
See how weighing credibility on a (skewed) left/right bias metric waters this down? Both of these examples would get high credibility.
On top of that, none of this impacts this community at all. It could be a problem if the standard here was ‘highest’ ratings exclusively, but it isn’t.
That’s a fair point and I did state in my original post that despite my own feelings I’d be open to something like this if the community had been more involved in the process of choosing one/deciding one is necessary and also if we had the bots post clearly call out it’s biases, maybe an explanation of its methodology and the inherent risks in it.
The way it’s been pushed from the mod first without polling the community and seeing the reaction to criticism some of which was constructive is my main issue here really.
This is only true specifically when you’re thinking about it as a great source can’t have its credibility rating lowered. A not great factual source can get a high credibility rating if it’s deemed centrist enough which again is arbitrary based on the (effectively) 1 guys personal opinion.
The impact either way is slight. I’m sure you could find a few edge cases you could make an argument about because no methodology is perfect, but each outlier represents a vanishingly small (~0.01%) amount of their content. When you look at rigorous research on the MBFC dataset though, the effect just isn’t really there. Here’s another study that concludes that the agreement between bias-monitoring organizations is so high that it doesn’t matter which one you use. I’ve looked and I can’t find research that finds serious bias or methodological problems. Looking back at the paper I posted in my last comment, consensus across thousands of news organizations is just way too high to be explainable by chance. If it was truly arbitrary as people often argue, MBFC would be an outlier. If all the methodologies were bad, the results would be all over the map because there are many more ways to make a bad methodology than a good one. What the research says is that if one methodology is better than the others, it isn’t much better.
Again, I think you make a really good argument for why MBFC and sites like it shouldn’t be used in an extreme, heavy-handed way. But it matters if it has enough precision for our purposes. Like, if I’m making bread, I don’t need a scale that measures in thousandths of a gram. A gram scale is fine. I could still churn out a top-shelf loaf with a scale that measures in 10-gram units. This bot is purely informational. People are reacting like it’s a moderation change but it isn’t – MBFC continues to be one resource among many that mods use to make decisions. Many react as though MBFC declares a source either THE BEST or THE WORST (I think a lot of those folks aren’t super great with nuance) but what it mostly does is say ‘this source is fine but there’s additional info or context worth considering.’ Critics often get bent out of shape about the ranking but almost universally neglect the fact that, if you click that link, there’s a huge report on each source that provides detailed info about their ownership history, funding model, publishing history, biases, and the press freedom of the country they’re in. Almost every time, there are reasonable explanations for the rankings in the report. I have not once ever seen someone say, like, ‘MBFC says that this is owned by John Q. Newspaperman but it’s actually owned by the Syrian government,’ or ‘they claim that they had a scandal with fabricated news but that never happened’. Is there a compelling reason why we’re worse off knowing that information? If you look at the actual reports for Breitbart and the Kyiv Independent, is there anything in there that we’re better off not knowing?
Like I kinda said in my last paragraphs you’ve got fair points that it may be good enough for what it’s being used for here (despite it’s clear biases) since it’s not being used to disallow posts. Although other commenters have said it has a pro-Zionist bias as well which is honestly more concerning than things I’ve pointed out. Haven’t had time to check beyond the ADL one.
Overall my main issue is the community wasn’t really asked if one was desired, which one should be used, how it should be used, etc. Because of that and the lack of good response by the poster I’ve already decided to follow other world news communities instead of this one.
