The Chronicle of Higher Education has a long read on what appears to be a case of straight-out research fraud to which the editor of the journal is completely indifferent.
https://www-chronicle-com/interactives/20190924-Criminology
A quick synopsis--a prominent social-science researcher publishes a series of papers with co-authors that have blatantly suspicious numbers. Somebody catches this and reports it to the authors. The co-authors get suspicious and ask for more information from the lead author.
No way! Stop attacking me! Appeals to the EIC of the journal.
Faked data? Whatever. We don't talk about that. Co-author publishes retraction (https://osf.io/preprints/socarxiv/9b2k3/) elsewhere. Potential repercussions ensue, for the coauthors trying to correct the record.
Some of the claims that astound me:
- Critically analyzing a paper after it is published is seen as improper behavior: "blood sport."
- Co-authors have no expectation of seeing the raw data associated with the research they are writing about.
- EIC: "I don't even quite know what retraction is. I imagine that it could occur."
The social norms in this field seem completely opposite of those that support real science. Publication is when the real critical analysis begins. The quality of the data need to meet really high standards; ideally they are published as a supplement or at a repository. Co-authors must sign off that they stand behind the data. Retraction is a normal, if unpleasant, act by an editor-in-chief.
After this, I can't imagine any serious scholar submitting papers to
Criminology. While it has been the flagship journal, its editor unambiguously condones publication of fabricated reports.
Quote from: Hibush on September 25, 2019, 03:12:05 AM
- Co-authors have no expectation of seeing the raw data associated with the research they are writing about.
This stands out to me as being well outside norms. "Raw" may come in various forms, but did no one see preliminary results and have extended discussions well before the final paper was submitted?
My experience is one brings a couple graphs and tables to the meeting, we discuss as a group what else we'd like to see, and then we iterate on that for a weeks/months before we write the paper. Even during the writing, it's pretty common for someone to ask for more analysis or point out that a table needs to be checked because a couple numbers seem off now that we're really down in the weeds.
I'm much more familiar with authors being embarrassed over submitting tables with typos or clearly the wrong graph (d'oh, the problems with naming everything using something like uniqueName_version2_date_afterStevefixedthelegend_afterMarthafixedthesymbols and then not double checking that it's not uniqueName_version2_date_afterStevefixedthelegend_afterMarthafixedthesymbols_updateforNewTemp) or pushing hard on "that's a whole different study; go ahead and run your own study if you'd like to focus on that aspect of the analysis" than "don't attack me.
0_o
Then again, is it all that surprising, given how much of forensics is utter bunkum? (Or is criminology substantially different?)
Quote from: Parasaurolophus on September 25, 2019, 07:52:19 AM
0_o
Then again, is it all that surprising, given how much of forensics is utter bunkum? (Or is criminology substantially different?)
Apparently most of the authors identify as sociologists. The paper is about behavioral impacts of policing practices. Do forumites have insight on the relationship among these subdisciplines and the relative prevalence of bunkum?
Quote from: Hibush on September 25, 2019, 08:39:51 AM
Quote from: Parasaurolophus on September 25, 2019, 07:52:19 AM
0_o
Then again, is it all that surprising, given how much of forensics is utter bunkum? (Or is criminology substantially different?)
Apparently most of the authors identify as sociologists. The paper is about behavioral impacts of policing practices. Do forumites have insight on the relationship among these subdisciplines and the relative prevalence of bunkum?
Psychologist joke:
What do you call a psychology major who can't pass statistics?
A sociology major.
Mean and somewhat unfair (there are some sociologists who are
highly quantitative), and of course all fields have some bunkum, but the journal editor's response is quite shocking to me for any field-- has the guy seriously not encountered any mention of the replication crisis at all?
I hadn't seen this article (thanks!) but am wondering if this is what James (Heathers) and Nick (Brown) had alluded to a few months ago. I follow their work pretty closely, and each new case still manages to amaze. I agree that it's surprising to see a journal editor (of a flagship journal?!) who seems to be ignorant of the movements toward correcting flawed science, especially when papers have been highly cited. His preferred way of handling it seems like an old boys' club approach to me. Foregoing transparency is, I think, the wrong path to go down these days (finally). I'm sure we all make mistakes despite our best efforts, but the resistance to sharing data and figuring out what might have gone wrong (if anything) is understandable to some extent but also unacceptable if the data still exist. The fact that some researchers appear to have numerous papers with the same issues is very discouraging, and I don't see how "data thugs" is an accurate term for those who develop and apply methods to examine data more carefully in the service of identifying major flaws that compromise the integrity of the findings that so many researchers apparently base their own research on. Beyond that, the idea that a co-author is getting pushback about daring to request to see the data that underlie the paper his own name and reputation are tied to is absolutely ridiculous. I don't know the researchers involved in this work, but from the article it doesn't seem as though he is trying to perform some character assassination on his former mentor.
Quote from: ab_grp on September 25, 2019, 11:52:41 AM
I hadn't seen this article (thanks!) but am wondering if this is what James (Heathers) and Nick (Brown) had alluded to a few months ago. I follow their work pretty closely, and each new case still manages to amaze. I agree that it's surprising to see a journal editor (of a flagship journal?!) who seems to be ignorant of the movements toward correcting flawed science, especially when papers have been highly cited. His preferred way of handling it seems like an old boys' club approach to me.
The old-boys club culture can be shocking, at least to those of us in fields where the data have power and the current models are continuously updated. The citation isn't coming to me now (possibly Richard Harris'
Rigor Mortis), but one example was from psychology where the culture was to publish your grand theory soon after your PhD, and then stick with it through your career. Your subsequent publications should be consistent with your grand theory (carefully avoiding any experiments that might falsify it). Then the kicker is that nobody else is supposed to challenge your grand theory until you are dead. Maybe that is the culture the
Criminology EIC came from?
Can we all feel empowered to stamp that stuff out?
Quote from: Hibush on September 25, 2019, 12:25:48 PM
Quote from: ab_grp on September 25, 2019, 11:52:41 AM
I hadn't seen this article (thanks!) but am wondering if this is what James (Heathers) and Nick (Brown) had alluded to a few months ago. I follow their work pretty closely, and each new case still manages to amaze. I agree that it's surprising to see a journal editor (of a flagship journal?!) who seems to be ignorant of the movements toward correcting flawed science, especially when papers have been highly cited. His preferred way of handling it seems like an old boys' club approach to me.
The old-boys club culture can be shocking, at least to those of us in fields where the data have power and the current models are continuously updated. The citation isn't coming to me now (possibly Richard Harris' Rigor Mortis), but one example was from psychology where the culture was to publish your grand theory soon after your PhD, and then stick with it through your career. Your subsequent publications should be consistent with your grand theory (carefully avoiding any experiments that might falsify it). Then the kicker is that nobody else is supposed to challenge your grand theory until you are dead. Maybe that is the culture the Criminology EIC came from?
Can we all feel empowered to stamp that stuff out?
Huh, that's not at all recognizable as psychology as the field that I know. What era and subfield was that supposed to be describing? We tend to be pretty into knocking holes in one another's theories, and getting much better about not getting too attached to our own theories. Plus, most areas don't go in for grand theories all that much these days. To our credit, I think we have addressed issues of rigor and reproducibility pretty head on in the last five years or so-- there has been a huge amount of cultural change around pre-registration, linking data and code to papers, etc.
Quote from: Puget on September 25, 2019, 02:26:47 PM
Quote from: Hibush on September 25, 2019, 12:25:48 PM
Quote from: ab_grp on September 25, 2019, 11:52:41 AM
I hadn't seen this article (thanks!) but am wondering if this is what James (Heathers) and Nick (Brown) had alluded to a few months ago. I follow their work pretty closely, and each new case still manages to amaze. I agree that it's surprising to see a journal editor (of a flagship journal?!) who seems to be ignorant of the movements toward correcting flawed science, especially when papers have been highly cited. His preferred way of handling it seems like an old boys' club approach to me.
The old-boys club culture can be shocking, at least to those of us in fields where the data have power and the current models are continuously updated. The citation isn't coming to me now (possibly Richard Harris' Rigor Mortis), but one example was from psychology where the culture was to publish your grand theory soon after your PhD, and then stick with it through your career. Your subsequent publications should be consistent with your grand theory (carefully avoiding any experiments that might falsify it). Then the kicker is that nobody else is supposed to challenge your grand theory until you are dead. Maybe that is the culture the Criminology EIC came from?
Can we all feel empowered to stamp that stuff out?
Huh, that's not at all recognizable as psychology as the field that I know. What era and subfield was that supposed to be describing? We tend to be pretty into knocking holes in one another's theories, and getting much better about not getting too attached to our own theories. Plus, most areas don't go in for grand theories all that much these days. To our credit, I think we have addressed issues of rigor and reproducibility pretty head on in the last five years or so-- there has been a huge amount of cultural change around pre-registration, linking data and code to papers, etc.
That is very good to hear!
I don't have access to the CHE article but I was curious so I went ahead and read the retraction paper published by one of the co-authors. It lists so many issues, and of such gravity, that it is quite hard to find a benign explanation. But even if it were just a case of (extreme) sloppiness, I cannot imagine that an editor would choose not to react to such an inquiry.
Reviving since this seems to be the closest thread on the topic....
A French oceanographic institute is having trouble getting its research correctly attributed and cited....
https://retractionwatch.com/2022/02/03/french-ocean-institute-goes-public-about-authors-who-forged-their-researchers-names/
...and having certain, shall we say, "fictitious" authors' names removed...
M.
Quote from: mamselle on February 05, 2022, 01:27:40 PM
Reviving since this seems to be the closest thread on the topic....
A French oceanographic institute is having trouble getting its research correctly attributed and cited....
https://retractionwatch.com/2022/02/03/french-ocean-institute-goes-public-about-authors-who-forged-their-researchers-names/
...and having certain, shall we say, "fictitious" authors' names removed...
M.
Wow.
...Is there a point to doing this? Especially if it's just random people who don't work in the field, rather than superstars?
It is important to be aware that the predatory publishing world is big. These papers are in a junk journal from a predatory publisher, Scholink.
Think of it like spam email. The most recent report I read estimated spam as being >60% of total emails. Thankfully we don't see most of them due to good filters. Those filters don't exist for predatory publishers. They will have to be put in place for legitimate publising to thrive.
Quote from: Hibush on February 06, 2022, 05:08:40 AM
It is important to be aware that the predatory publishing world is big. These papers are in a junk journal from a predatory publisher, Scholink.
Think of it like spam email. The most recent report I read estimated spam as being >60% of total emails. Thankfully we don't see most of them due to good filters. Those filters don't exist for predatory publishers. They will have to be put in place for legitimate publising to thrive.
In what way do you think filters don't exist? At least in the sciences, indexing databases like PubMed, Web of Science, Scopus etc. service as pretty effective filters-- they don't include scam journals, so the average researcher searching for journal articles will never come across those from scam journals.
My impression is that scam journals don't actually care if anyone reads the articles-- their business model is to charge authors to publish.
Quote from: Hibush on February 06, 2022, 05:08:40 AM
It is important to be aware that the predatory publishing world is big. These papers are in a junk journal from a predatory publisher, Scholink.
Think of it like spam email. The most recent report I read estimated spam as being >60% of total emails. Thankfully we don't see most of them due to good filters. Those filters don't exist for predatory publishers. They will have to be put in place for legitimate publising to thrive.
I still don't understand the point of putting real names on a fake paper. I mean, I get the point of fake papers: the "journal" gets money and the "researcher" gets an "article" to put on their CV for tenure, promotion, etc. But what is gained by putting the name of somebody uninvolved in a different discipline? Is it just because most papers in the field need multiple authors, so these are put on just to look right?
In terms of filters, it seems that if this plan has any chance of success, it is precisely because there is a filter for this sort of stuff. Just as I don't know if my name is being used to sell viagra online because I don't see most of those advertisements, I wouldn't really know if my name is appended to a fake paper in a different field because I don't read those journals, my university doesn't subscribe to them, etc.
Quote from: Puget on February 06, 2022, 06:57:13 AM
Quote from: Hibush on February 06, 2022, 05:08:40 AM
It is important to be aware that the predatory publishing world is big. These papers are in a junk journal from a predatory publisher, Scholink.
Think of it like spam email. The most recent report I read estimated spam as being >60% of total emails. Thankfully we don't see most of them due to good filters. Those filters don't exist for predatory publishers. They will have to be put in place for legitimate publising to thrive.
In what way do you think filters don't exist? At least in the sciences, indexing databases like PubMed, Web of Science, Scopus etc. service as pretty effective filters-- they don't include scam journals, so the average researcher searching for journal articles will never come across those from scam journals.
My impression is that scam journals don't actually care if anyone reads the articles-- their business model is to charge authors to publish.
I mostly use Google scholar, and it is unfiltered. The problem with filters is that they filter out too much. Even if something is published in a sketchy journal you should still acknowledge the claims.
For example, you could run into issues if you make a claim as being novel if it has already been made elsewhere, even if you don't like the venue. The reality is that predatory journals range from complete garbage to ones that are just not yet listed. Likewise, they have papers that are junk, as well as some decent ones that authors published there for whatever reason. I assume people sometimes.publish there because some with names similar to well established journals, and in my field predatory open access journals are cheaper than well established OA journals. Regardless, you can't just ignore claims or ideas based on the publication venue.
On the other side of the question, at least some of the offenders are caught:
https://retractionwatch.com/2022/02/10/more-than-100-of-an-anesthesiologists-papers-retracted/
M.
Quote from: Kron3007 on February 10, 2022, 07:07:25 PM
Quote from: Puget on February 06, 2022, 06:57:13 AM
Quote from: Hibush on February 06, 2022, 05:08:40 AM
It is important to be aware that the predatory publishing world is big. These papers are in a junk journal from a predatory publisher, Scholink.
Think of it like spam email. The most recent report I read estimated spam as being >60% of total emails. Thankfully we don't see most of them due to good filters. Those filters don't exist for predatory publishers. They will have to be put in place for legitimate publising to thrive.
In what way do you think filters don't exist? At least in the sciences, indexing databases like PubMed, Web of Science, Scopus etc. service as pretty effective filters-- they don't include scam journals, so the average researcher searching for journal articles will never come across those from scam journals.
My impression is that scam journals don't actually care if anyone reads the articles-- their business model is to charge authors to publish.
I mostly use Google scholar, and it is unfiltered. The problem with filters is that they filter out too much. Even if something is published in a sketchy journal you should still acknowledge the claims.
For example, you could run into issues if you make a claim as being novel if it has already been made elsewhere, even if you don't like the venue. The reality is that predatory journals range from complete garbage to ones that are just not yet listed. Likewise, they have papers that are junk, as well as some decent ones that authors published there for whatever reason. I assume people sometimes.publish there because some with names similar to well established journals, and in my field predatory open access journals are cheaper than well established OA journals. Regardless, you can't just ignore claims or ideas based on the publication venue.
I'm going to disagree with this. I would treat any work published in a predatory as being non-peer reviewed and suspect until proven otherwise. I would certainly not cite it. I wouldn't trust science done by someone who doesn't know what the legitimate journals in their field are either.
Quote from: Puget on February 10, 2022, 07:16:53 PM
Quote from: Kron3007 on February 10, 2022, 07:07:25 PM
Quote from: Puget on February 06, 2022, 06:57:13 AM
Quote from: Hibush on February 06, 2022, 05:08:40 AM
It is important to be aware that the predatory publishing world is big. These papers are in a junk journal from a predatory publisher, Scholink.
Think of it like spam email. The most recent report I read estimated spam as being >60% of total emails. Thankfully we don't see most of them due to good filters. Those filters don't exist for predatory publishers. They will have to be put in place for legitimate publising to thrive.
In what way do you think filters don't exist? At least in the sciences, indexing databases like PubMed, Web of Science, Scopus etc. service as pretty effective filters-- they don't include scam journals, so the average researcher searching for journal articles will never come across those from scam journals.
My impression is that scam journals don't actually care if anyone reads the articles-- their business model is to charge authors to publish.
I mostly use Google scholar, and it is unfiltered. The problem with filters is that they filter out too much. Even if something is published in a sketchy journal you should still acknowledge the claims.
For example, you could run into issues if you make a claim as being novel if it has already been made elsewhere, even if you don't like the venue. The reality is that predatory journals range from complete garbage to ones that are just not yet listed. Likewise, they have papers that are junk, as well as some decent ones that authors published there for whatever reason. I assume people sometimes.publish there because some with names similar to well established journals, and in my field predatory open access journals are cheaper than well established OA journals. Regardless, you can't just ignore claims or ideas based on the publication venue.
I'm going to disagree with this. I would treat any work published in a predatory as being non-peer reviewed and suspect until proven otherwise. I would certainly not cite it. I wouldn't trust science done by someone who doesn't know what the legitimate journals in their field are either.
Concepts don't need data to be original. If someone publishes a hypothesis, you can no longer publish that same hypothesis as being original, even if their actual study was flawed. If you fail to cite their original claim, even if it is published in a sketchy journal, that is on you.
As for not being able to tell, Isaw one preditory journal with the exact same name as a real, well respected, journal. I only knew because I was reading an article and was wondering how that journal published such poor research. When I searched the doi, it took me to the fake journal. They are clever and many older, less tech savvy, professors who do good research could fall for it.
Don't take me wrong, I think predatory journals are one of the banes of modern research and a critical problem. However, a proper literature search should not exclude them entirely. As you don't trust an academic who dosn't know the difference, I could argue that I don't trust one who can't evaluate the research on its merits and relies on flawed filters.
I will also mention that some publishers (ie MDPI) are listed, but publish a lot of garbage. Are they preditory? Kind of...
Quote from: Puget on February 10, 2022, 07:16:53 PM
I'm going to disagree with this. I would treat any work published in a predatory as being non-peer reviewed and suspect until proven otherwise. I would certainly not cite it. I wouldn't trust science done by someone who doesn't know what the legitimate journals in their field are either.
I have recently encountered several decent articles published in MDPI journals by people I personally know.
While I haven't asked them directly, I suspect their decision to publish in these journals was driven by the publication speed.
For people working solely in the industry, waiting a year from submission to publication can be very problematic: one may change job, thus, losing access to the original data / co-authors; long wait times increase probability of not getting client's approvals for the final version (if your original contact there moves on); etc.
So, ignoring these publications completely would limit one's sources to people in stable academic/governmental positions and exclude many actual practitioners.
Quote from: Durchlässigkeitsbeiwert on February 11, 2022, 04:06:24 AM
Quote from: Puget on February 10, 2022, 07:16:53 PM
I'm going to disagree with this. I would treat any work published in a predatory as being non-peer reviewed and suspect until proven otherwise. I would certainly not cite it. I wouldn't trust science done by someone who doesn't know what the legitimate journals in their field are either.
I have recently encountered several decent articles published in MDPI journals by people I personally know.
While I haven't asked them directly, I suspect their decision to publish in these journals was driven by the publication speed.
For people working solely in the industry, waiting a year from submission to publication can be very problematic: one may change job, thus, losing access to the original data / co-authors; long wait times increase probability of not getting client's approvals for the final version (if your original contact there moves on); etc.
So, ignoring these publications completely would limit one's sources to people in stable academic/governmental positions and exclude many actual practitioners.
I have published a couple articles in MDPI journals based on similar observations. Many respected colleagues publish in them and there is good work published there.
However, I did not find their peer review as rigorous as most journals and am confident that poor quality work would also make it through their process without much issue. The feedback I got was very superficial and editorial in nature. Speed comes at a price....
They charge over a thousand dollars per article and just churn them out. They spam me to guest host special issue all the time, offering discounts and free publication. I agree with rewarding reviewers and editors for their time, but question their quality and operations.
Quote from: Kron3007 on February 11, 2022, 04:45:34 AM
Quote from: Durchlässigkeitsbeiwert on February 11, 2022, 04:06:24 AM
Quote from: Puget on February 10, 2022, 07:16:53 PM
I'm going to disagree with this. I would treat any work published in a predatory as being non-peer reviewed and suspect until proven otherwise. I would certainly not cite it. I wouldn't trust science done by someone who doesn't know what the legitimate journals in their field are either.
I have recently encountered several decent articles published in MDPI journals by people I personally know.
While I haven't asked them directly, I suspect their decision to publish in these journals was driven by the publication speed.
For people working solely in the industry, waiting a year from submission to publication can be very problematic: one may change job, thus, losing access to the original data / co-authors; long wait times increase probability of not getting client's approvals for the final version (if your original contact there moves on); etc.
So, ignoring these publications completely would limit one's sources to people in stable academic/governmental positions and exclude many actual practitioners.
I have published a couple articles in MDPI journals based on similar observations. Many respected colleagues publish in them and there is good work published there.
However, I did not find their peer review as rigorous as most journals and am confident that poor quality work would also make it through their process without much issue. The feedback I got was very superficial and editorial in nature. Speed comes at a price....
They charge over a thousand dollars per article and just churn them out. They spam me to guest host special issue all the time, offering discounts and free publication. I agree with rewarding reviewers and editors for their time, but question their quality and operations.
Nothing you've written here makes me change my mind one bit. In fact, you're making my case for me.
I would never claim originally of a hypothesis, for myself or anything I cite-- that's sort of silly, as someone somewhere may have had the same thought earlier. A hypothesis means very little-- show me the data.
And yes, I can evaluate work quite well thank you very much, but unless I've analyzed the data myself I'm not going to trust what a paper claims that is published in a predatory journal by people that don't know better or don't care (even if it has the same title, surely the unfamiliar submission portal etc. would be a tip-off? I just don't think it is very hard.)
Quote from: Puget on February 11, 2022, 06:29:06 AM
Quote from: Kron3007 on February 11, 2022, 04:45:34 AM
Quote from: Durchlässigkeitsbeiwert on February 11, 2022, 04:06:24 AM
Quote from: Puget on February 10, 2022, 07:16:53 PM
I'm going to disagree with this. I would treat any work published in a predatory as being non-peer reviewed and suspect until proven otherwise. I would certainly not cite it. I wouldn't trust science done by someone who doesn't know what the legitimate journals in their field are either.
I have recently encountered several decent articles published in MDPI journals by people I personally know.
While I haven't asked them directly, I suspect their decision to publish in these journals was driven by the publication speed.
For people working solely in the industry, waiting a year from submission to publication can be very problematic: one may change job, thus, losing access to the original data / co-authors; long wait times increase probability of not getting client's approvals for the final version (if your original contact there moves on); etc.
So, ignoring these publications completely would limit one's sources to people in stable academic/governmental positions and exclude many actual practitioners.
I have published a couple articles in MDPI journals based on similar observations. Many respected colleagues publish in them and there is good work published there.
However, I did not find their peer review as rigorous as most journals and am confident that poor quality work would also make it through their process without much issue. The feedback I got was very superficial and editorial in nature. Speed comes at a price....
They charge over a thousand dollars per article and just churn them out. They spam me to guest host special issue all the time, offering discounts and free publication. I agree with rewarding reviewers and editors for their time, but question their quality and operations.
Nothing you've written here makes me change my mind one bit. In fact, you're making my case for me.
I would never claim originally of a hypothesis, for myself or anything I cite-- that's sort of silly, as someone somewhere may have had the same thought earlier. A hypothesis means very little-- show me the data.
And yes, I can evaluate work quite well thank you very much, but unless I've analyzed the data myself I'm not going to trust what a paper claims that is published in a predatory journal by people that don't know better or don't care (even if it has the same title, surely the unfamiliar submission portal etc. would be a tip-off? I just don't think it is very hard.)
Yes, poor wording on my part, but If you are writing an intro or discussion section and claim/imply something is novel or untested (otherwise why would you do the study...), when similar experiments have been published in a predatory journal, you have failed in due diligence. In my opinion, it is your role to acknowledge the previous work and highlight flaws or limitations that justify your work.
Personally, my students submit their papers so I generally don't see the interface directly. If they tell me they are submitting to journal X, I assume it is the real one (generally a safe assumption) You can judge all you want, but many solid researchers have ended up publishing in questionable journals for various reasons (this is on them).
Finally, how do you deal with questionable journals that are listed, but are known to have issues (ie MDPI series)? How do you define a predatory journal? I have seen garbage work in nature, and good work in MDPI journals. I see good researchers publishing in MDPI journals that I know from experience have substandard peer review. Their work is still good, but the journal fails the smell test and also have low quality papers
I just think you are oversimplifying a complex problem, but we will have to agree to disagree