John Lott and research about firearms

[YukonGlocker]

The problems with peer-review apply to all objects of study, not only crime and guns. This is why social science is leading all areas of science in dealing with those issues by focusing on training people correctly to begin with, removing publication biases, and rewarding good science (regardless of the outcome) instead of headline newsworthy crappy science. Social science has led the change to open science, including pre-registration of studies, open methods, open data, open code for all analyses, then open and public peer-review. Every single study (peer-reviewed, or not) needs to be fully vetted by people that have the skills to identify problems, whether it's a crime-related study or a cancer-related study. There are a lot of poor studies that make it out of peer-review. But there are lot of great studies that make it out too (or, aren't peer-reviewed in a traditional way, for example blogs that are publicly reviewed and edited into remarkable pieces of information because they are hammered into greatness by great scientists intensely critiquing each other). And guess what, some of the brightest science critics around have found major issues with Lott's studies, and Lott refuses to allow access to data or code (for independent analysis), and refuses to change things that are clearly wrong. In short, Lott isn't being scientific, and that's why he and much of his work have become a joke in the scientific community.

 

[Glocktogo]

The problems with peer-review apply to all objects of study, not only crime and guns. This is why social science is leading all areas of science in dealing with those issues by focusing on training people correctly to begin with, removing publication biases, and rewarding good science (regardless of the outcome) instead of headline newsworthy crappy science. Social science has led the change to open science, including pre-registration of studies, open methods, open data, open code for all analyses, then open and public peer-review. Every single study (peer-reviewed, or not) needs to be fully vetted by people that have the skills to identify problems, whether it's a crime-related study or a cancer-related study. There are a lot of poor studies that make it out of peer-review. But there are lot of great studies that make it out too (or, aren't peer-reviewed in a traditional way, for example blogs that are publicly reviewed and edited into remarkable pieces of information because they are hammered into greatness by great scientists intensely critiquing each other). And guess what, some of the brightest science critics around have found major issues with Lott's studies, and Lott refuses to allow access to data or code (for independent analysis), and refuses to change things that are clearly wrong. In short, Lott isn't being scientific, and that's why he and much of his work have become a joke in the scientific community.

If they don't have access to the raw data OR the formulas, how do they know its flawed? By what standards did they determine that the data was corrupt? Science critics should be restricted to critiquing the science, not the results. Just because you're a science critic, doesn't mean you're unbiased.

 

[vvvvvvv]

If they don't have access to the raw data OR the formulas, how do they know its flawed?

Research must be able to be duplicated to determine it's validity. If the data is not available and the processes used to test that data are opaque, how can anyone prove the research is valid?

By refusing to allow access to data and processes that were used to reach a conclusion, a researcher is pretty much saying "here are my results - trust me."

 

[Glocktogo]

Research must be able to be duplicated to determine it's validity. If the data is not available and the processes used to test that data are opaque, how can anyone prove the research is valid?

By refusing to allow access to data and processes that were used to reach a conclusion, a researcher is pretty much saying "here are my results - trust me."

So you're confirming that they can't realistically find problems with his work or prove he's "clearly wrong"? That they"re not privy to the important data, so they're assuming he has it wrong? If they can't prove its valid, then they automatically consider it flawed?

That doesn't sound like solid science to me.

 

[Dale00]

...In short, Lott isn't being scientific, and that's why he and much of his work have become a joke in the scientific community.

Are pro-gun control researchers who refuse to share their raw data, like University of Alabama researcher Adam Lankford (see my post above), likewise, "a joke in the scientific community"?

If they are not, this tells us that there is a large (perhaps even massive) amount of bias in the social science community on the topic of the impact of gun regulations and laws. If this is the case, social science researchers are disreputable and their research findings are not worthy of respect.

 

[Glocktogo]

Are pro-gun control researchers who refuse to share their raw data, like University of Alabama researcher Adam Lankford (see my post above), likewise, "a joke in the scientific community"?

If they are not, this tells us that there is a large (perhaps even massive) amount of bias in the social science community on the topic of the impact of gun regulations and laws. If this is the case, social science researchers are disreputable and their research findings are not worthy of respect.

Well we have to assume they're right and Lott is wrong, because they're more sciencey than he is! Lol

 

[YukonGlocker]

If they don't have access to the raw data OR the formulas, how do they know its flawed? By what standards did they determine that the data was corrupt? Science critics should be restricted to critiquing the science, not the results. Just because you're a science critic, doesn't mean you're unbiased.

There are many problems with methodology, analyses, statistics, and interpretations of results that can be determined without access to data. I'm not sure what you mean by "corrupt" data...if you'll post a specific example, I'll check it out. There are no restrictions on vetting studies...every aspect from conception to final results/interpretations are open to scrutiny. That's precisely how we correct mistakes, learn where and how we were wrong, and then produce something that's more accurate than the last attempt.

 

[rawhide]

Last fall I did quite a bit of reading on the topic of Lott's research being discredited spending several hours reading the anti Lott position. After this amount of time I don't readily recall all the specifics and don't feel compelled to read them again. But, what I took away from reading the criticisms of Lott's research and the back and forth of those commenting on the research was that the de-bunking of Lott was pretty suspect. I even found a couple researchers that while attempting to refute Lott's research found that they agreed with him and changed their own position on guns and crime.

 

[YukonGlocker]

So you're confirming that they can't realistically find problems with his work or prove he's "clearly wrong"? That they"re not privy to the important data, so they're assuming he has it wrong? If they can't prove its valid, then they automatically consider it flawed?

That doesn't sound like solid science to me.

Again, most problems don't require the raw data. However, is somebody does need the data (could be for a variety of reasons), you'd better have a damn good reason for not allowing access to it. The best practicing journals are now *requiring* you to publish the data openly when you submit an article for publication. And, for the ones that don't, most governing bodies (e.g., Association for Psychological Science) have ethical standards that require you to share your data if requested by anyone (with few clear exceptions).

 

[YukonGlocker]

Are pro-gun control researchers who refuse to share their raw data, like University of Alabama researcher Adam Lankford (see my post above), likewise, "a joke in the scientific community"?

If they are not, this tells us that there is a large (perhaps even massive) amount of bias in the social science community on the topic of the impact of gun regulations and laws. If this is the case, social science researchers are disreputable and their research findings are not worthy of respect.

I haven't looked into the Lankford case. Does it say why he wouldn't share the data?