Article Needs to be Retracted (Not mine), Editors not Responding?

[mcj1218]

I recently discovered/noticed a situation where there is a clear need for an article in a peer-reviewed journal to be retracted. I contacted the editor (copied the assistant editor on the email) with my detailed explanation, noting that one of the coauthors communicated to me that he would contact the lead author and instruct her to email the editor to address this situation. I have good reason to doubt that the authors contacted the editor at all. I waited about a week after I was told that the editors were contacted to send the email. After about three weeks, I had no response and sent a follow-up yesterday.

I have more details about why the paper needs to be retracted below, but my major question is pretty straightforward. What can/should I do in the event that I get no response from the editorial board and there is no other indication that anything has been done about this?

The paper:

The fatal flaw of this paper involved a serious data issue. The problems are noticeable from reading the article, but I also had access to the data set used in the study and this made things look much worse. I was originally approached by one of the coauthors (there were four) to potentially do some data analysis for a new paper. I evaluated the data before reading the article.

There were 26 Likert-type survey items that represented a major part of the analyses, each having five possible response options coded 0-4 (based on the codebook, survey instrument and descriptions in the article). The article shows a total sample of over 300, but all subsequent analyses used no more than 170 cases (none were supposed to be from a subsample). There was no explanation for the drastic gap between the sample as described and the drastically reduced sample from analyses.

The egregious data problems: For each of the 26 variables in the original data set, a substantial percentage of cases were scored "5" or "6" (as opposed to 0-4 and independent of identified missing data). There is no justification for these two values to appear at all. These two scores represented more than 30% of the total sample for all but three of the measures and several represented over 40%. There was no accounting for this anywhere. At least one of the authors was in charge of collecting the data, and he could/would not explain how this occurred.

About half of the sample is excluded because scores on key variables seemingly came from nowhere. The results have no meaning. If this isn't an obvious retraction, what is? The paper has been cited over 20 times.

I fear this was more information than anyone wanted. Nevertheless, I'm happy to provide more details if desired. If anyone wants to discuss how this thing ever got past an initial data screening and a peer review, that works too.

[Puget]

This is bad, but maybe not malicious -- it's a busy time of the semester, and being an academic editor is a volunteer service position in most cases, so i'd give them a bit more time to respond. They will need to investigate themselves. In your position, if I still hadn't heard back in a month I think I would start contacting other editors at the journal. You could also report it to the research integrity office at the lead author's institution, though that is a bit of a nuclear option.

As an aside, this is why I instill in students to always look at descriptives, including range, for all variables, and examine the bivariate correlation matrix, before proceeding with analyses. Too many people skip this and just press run on their analysis code, and then never ask themselves questions about things that seem odd  (like "Gee, why is my N so much lower than I expected?"), and stuff like this is the result.

[ocean2428]

Why not submit a formal comment based on your analysis? editor has to respond to that..

[mcj1218]

This is bad, but maybe not malicious -- it's a busy time of the semester, and being an academic editor is a volunteer service position in most cases, so i'd give them a bit more time to respond. They will need to investigate themselves. In your position, if I still hadn't heard back in a month I think I would start contacting other editors at the journal. You could also report it to the research integrity office at the lead author's institution, though that is a bit of a nuclear option.

As an aside, this is why I instill in students to always look at descriptives, including range, for all variables, and examine the bivariate correlation matrix, before proceeding with analyses. Too many people skip this and just press run on their analysis code, and then never ask themselves questions about things that seem odd  (like "Gee, why is my N so much lower than I expected?"), and stuff like this is the result.

Thanks for the advice. I think it may have started out very bad and at some point, someone must have noticed it. At that point, it may be a passive malice. I would be incredibly embarrassed to have my name on something like this. A tenured, well-published professor, an Associate Dean/Head of a Doctoral Program/Instructor for the program's PhD methods course, and a department chair all seemed fine with it.

[mcj1218]

Why not submit a formal comment based on your analysis? editor has to respond to that..

I am not quite sure what a formal comment is. I apologize for the ignorance, but haven't come across that sort of mechanism.

[Ruralguy]

We've all made mistakes, but this is pretty bad. Many red flags ignored, or at best, not explained.

So, this has already been reviewed and published? Not a very careful review. Or the reviewers were ignored?
Or maybe you were the reviewer?

Wait for it to go through the process. As with anything like this, once you report it, it's not "yours" and there is no obligation to report back to you  on the progress. Of course, if you are some sort of direct witness, other than just sort of noticing this, then they should be back in touch.

[mcj1218]

Thanks for the advice and here is an overlong response:

It apparently was reviewed, but I can't imagine any kind of competent, unbiased review would have immediately found this. I was definitely not a reviewer. I came on the scene well after this article was published with the least senior coauthor on the paper to look at the data for potential future papers (I knew very little about anything until I began looking at the data set). I noticed all of the red flags almost immediately, inquired about them to the would-be collaborator, and was unable to get any kind of explanation.

My involvement is as someone who has noticed egregious errors in published paper that have been essentially admitted to by at least two of the coauthors. The paper is being cited in other academic works as accurate research. I really think it is my responsibility to do whatever I can to have it retracted. There is probably much more published "PR" research in my than anyone would like to admit, which is a bit demoralizing in so many ways. I know of this one and there is no doubt about it.

---Additional notes: The first author is the reviews editor for the journal. The one coauthor who actually responded to my inquiries is problematic in a number of ways. He initially disclosed that he was paid as a private consultant to conduct the survey for a municipal LE agency, so he made money from this. He would not answer any of my follow up questions about the data collection once I was aware of his involvement. There is the possibility that the editor was contacted by the authors and told not to respond to any potential correspondence from myself.

I will give the editors and authors plenty of time to do the right thing. If I am ignored or if they dismiss the evidence, I'll take it to a broader audience.

[ocean2428]

for example, Science will give you an option of technical comment

https://www.sciencemag.org/journal-department/technical-comment

Nature

https://www.nature.com/nature/for-authors/matters-arising

PRL

https://journals.aps.org/prl/authors/comments-physical-review-letters

So if you have serious concerns then you can submit a comment to the journal that published the work. It will go through the review process.

Why not submit a formal comment based on your analysis? editor has to respond to that..

I am not quite sure what a formal comment is. I apologize for the ignorance, but haven't come across that sort of mechanism.

[jerseyjay]

I am neither in the sciences nor a journal editor, so take my view for what it is worth.

Assuming that everything said about the article is correct, I am not sure why the OP would expect a response for the journal editor other than, perhaps, an acknowledgement of the original communication.

First, it would seem to me that retraction, like peer review, could take a while. Presumably the journal has a process to deal with this, which I assume is somewhat involved. (I would assume something like: the editor has to be convinced that the paper is really flawed and not just the victim of somebody who hates one of the authors, and then the editor has to convince others on the editorial board that there is a problem with the paper, and then perhaps reach out to the authors and/or peer reviewers to find out what happened, and then there has to be a decision what to do, and then this has to be done. And it possible that the journal's publishers have a protocol that needs to be followed.) Each step along the way there may be conflicting opinions, not to mention people who are otherwise busy. So I would not expect this to move quickly, especially since a retraction is not something that is done carelessly.

Second, it would seem to me that unless the OP is one of the authors or one of the reviewers or a member of the editorial board, there is no reason to get back to him or her. Not only would they have no formal role in the process, but it is possible they have some interest in the paper's retraction (a rival, an enemy, an ex-spouse or ex-research partner). This does not mean that the OP's observations do not need to be taken seriously, but that they need to evaluated on their own merits by the editor.

Sometimes while teaching, I have a student tell me that another student has turned in plagiarized work. My response is always, thanks for letting me know and I will look into it. I never just take the student's word for it, nor do I inform the student whether I determine there has been plagiarism or what, if any, penalties I assess on the other student.  (If the other student is involved in the plagiarism, that would be another story.) Of course retraction is a public act, but that would seem to underline the motto, Measure twice and cut once.

Again, I am not in the sciences or a journal editor, so I could be off base.

[Ruralguy]

Jerseyjay,

I also have never been in an editorial position, but I'm inclined to agree with you.

Once you report something, its reported. You are only contacted again if someone needs you as a witness or you are some sort of direct victim or perpetrator. Otherwise, its thanks, but we'll see ya later.  I suppose you can ask at some point.

Also, is there a way to tactfully write an opposition paper that doesn't necessarily accuse, but which maybe says something like "Though seemingly Brown et all shows that betting on race horses causes cancer of the left pinky, a closer look at the data presented shows..." Of course, you can only do something like that based on what is presented (or glaringly not presented if something can be made of it).