Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
shakedown.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A community for live music fans with roots in the jam scene. Shakedown Social is run by a team of volunteers (led by @clifff and @sethadam1) and funded by donations.

Administered by:

Server stats:

290
active users

shakedown.social: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.4

#PreRegistration

1 post1 participant0 posts today
Public
Bjørn Sætrevik

Imagine you preregistered your study with directed hypotheses, but did not specify one-tailed or two-tailed testing. Result A is in the predicted direction but p = .05-.09. Result B is in the non-predicted direction but p < .05. Do you report:

· 0 people · Closed
#PreRegistration#OpenScience#NHST
Public
Center for Open Science

Preregistering your research is a great way to boost transparency and rigor. But which template is right for your study?

Join us on March 27 at 11 AM ET for a webinar on choosing the right preregistration template! We’ll explore OSF’s templates for different study types—including experiments, systematic reviews, qualitative methods, and more.

🔗 Register now: cos-io.zoom.us/webinar/registe

#OpenScience#Preregistration#Research
Public
UniversityofGroningenLibrary

🏆 17 eligible case studies were submitted for our #OpenResearch Award 2024. All eligible case studies will be highlighted here. Today:

#OpenSource #Science for Human-Machine Interaction: Helping machines understand and produce non-literal speech in multilingual contexts, by Xiyuan Gao:
🔗 rug.nl/research/openscience/op

All eligible case studies from 2024:
🔗 rug.nl/research/openscience/op

Xiyuan Gao
#machine#human#speech
Public *
Jan R. Boehnke

Signed. And featured in my #PeerReview seminar yesterday.

I don't think that this is only about #PreRegistration.

One of our #RCTs received this year a 2.5k word review which did not refer to the #registration, statistical analysis plan, protocol, nor submitted appendices 🤦🤷

Also extrapolating from the style of that review:
How much anger and frustration that person could have avoided.

Edit: Link (Google Doc):
docs.google.com/document/d/1Y9

Screenshot of a presentation slide, main content quoted below, and with a screenshot of a Tweet by Brendan Nylan promoting the initiative for Preregistration Transparency in peer review. Reviewing ALL relevant materials Are there appendices? Are there external resources associated with the manuscript (e.g., OSF files) and have you checked them as far as relevant for the manuscript? (e.g., at least whether they are there) Are the documents that pre-define the conduct of the study? Registration of the study or review? Pre-registration?
#NightshiftEditor#PublicationEthics#ResearchIntegrity
Public *
22

Going to do a controlled experiment that I’d like to preregister with y’all. I found octodon.social/@22/11080587453 by searching my backups for an article about continents but I have zero memory of that chart. I wonder what % of stuff I bookmarked (in my backups) or tooted I remember.

Some stuff of course yeah memorable articles I still invoke etc. but I bet if the things I backed up or tooted ~12 months ago I remember only 10% of them.

Will write a little script that queries my backups and randomly picks 25 entries between 9 and 15 months ago, and similarly 25 toots/threads. So my preregistration is that I’ll remember reading or writing 5 of these entries in total, with my 90% confidence interval being between 3 and 15.

(Caveat I’m not very good at benchmarking subjective probability so I can’t be sure how good of a map that 3 and 15 are. Sorry.)

Finally, I can imagine some ambiguity in answering “do I remember reading or writing…”. If I remember the title of the bookmark but not the content, or the context of the toot but not sending it, it counts as “remembering”. #preregistration

(Bibliography: on backups see octodon.social/@22/10935084284. On preregistration see Daniel Kahneman, “The crisis has been great for psychology. In terms of methodological progress, this has been the best decade in my lifetime. Standards have been tightened up, research is better, samples are larger. People pre-register their experimental plans and their plans for analysis” edge.org/adversarial-collabora)

the Octodon22 (@22@octodon.social)Attached: 1 image Happy summer from California: "Europe vs the United States: sunshine duration in hours per year" (originally via https://nitter.net/zachklein/status/1486586299507109894, 27 Jan 2022) Would love a global version of this! #dataviz #geography
Public
Emma Byrne PhD

Pandora's box is open: Charles University researchers demonstrate that #ChatGPT can create convincing, fraudulent "scientific" papers

jmir.org/2023/1/e46924/

Huge implications for #peerReviewed
#Science and (another) strong argument for #preregistration of studies

Journal of Medical Internet ResearchArtificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora’s Box Has Been OpenedBackground: Artificial intelligence (AI) has advanced substantially in recent years, transforming many industries and improving the way people live and work. In scientific research, AI can enhance the quality and efficiency of data analysis and publication. However, AI has also opened up the possibility of generating high-quality fraudulent papers that are difficult to detect, raising important questions about the integrity of scientific research and the trustworthiness of published papers. Objective: The aim of this study was to investigate the capabilities of current AI language models in generating high-quality fraudulent medical articles. We hypothesized that modern AI models can create highly convincing fraudulent papers that can easily deceive readers and even experienced researchers. Methods: This proof-of-concept study used ChatGPT (Chat Generative Pre-trained Transformer) powered by the GPT-3 (Generative Pre-trained Transformer 3) language model to generate a fraudulent scientific article related to neurosurgery. GPT-3 is a large language model developed by OpenAI that uses deep learning algorithms to generate human-like text in response to prompts given by users. The model was trained on a massive corpus of text from the internet and is capable of generating high-quality text in a variety of languages and on various topics. The authors posed questions and prompts to the model and refined them iteratively as the model generated the responses. The goal was to create a completely fabricated article including the abstract, introduction, material and methods, discussion, references, charts, etc. Once the article was generated, it was reviewed for accuracy and coherence by experts in the fields of neurosurgery, psychiatry, and statistics and compared to existing similar articles. Results: The study found that the AI language model can create a highly convincing fraudulent article that resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The AI-generated article included standard sections such as introduction, material and methods, results, and discussion, as well a data sheet. It consisted of 1992 words and 17 citations, and the whole process of article creation took approximately 1 hour without any special training of the human user. However, there were some concerns and specific mistakes identified in the generated article, specifically in the references. Conclusions: The study demonstrates the potential of current AI language models to generate completely fabricated scientific articles. Although the papers look sophisticated and seemingly flawless, expert readers may identify semantic inaccuracies and errors upon closer inspection. We highlight the need for increased vigilance and better detection methods to combat the potential misuse of AI in scientific research. At the same time, it is important to recognize the potential benefits of using AI language models in genuine scientific writing and research, such as manuscript preparation and language editing.
Public
petersuber

We often see social sciences try to emulate natural sciences. Here's a nice example of the reverse.
academic.oup.com/aje/advance-a

"Empirical evidence from the social sciences suggests such [#opendata, #opencode & #preregistration] practices are feasible, can improve analytic #reproducibility, and can reduce selective reporting. In academic #epidemiology, adoption of #openscience practices has been slower than in the social sciences [and could improve]."

OUP AcademicToward open and reproducible epidemiologyAbstract. Starting in the 2010s, researchers in the experimental social sciences rapidly began to adopt increasingly open and reproducible scientific practices.
#SocialSciences#NaturalSciences
Public
Mark Rubin

Exploratory hypothesis testing:

Nice to see that our article “Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests” (doi.org/10.1080/09515089.2022.), which was published 4 months ago, is the most read article in @PhilosophicalPsychology in the last 12 months: tandfonline.com/action/showMos

#Psychology
#PhilosophyOfScience
#PhilSci
#MetaScience
#Preregistration
#HypothesisTesting

Abstract Preregistration has been proposed as a useful method for making a publicly verifiable distinction between confirmatory hypothesis tests, which involve planned tests of ante hoc hypotheses, and exploratory hypothesis tests, which involve unplanned tests of post hoc hypotheses. This distinction is thought to be important because it has been proposed that confirmatory hypothesis tests provide more compelling results (less uncertain, less tentative, less open to bias) than exploratory hypothesis tests. In this article, we challenge this proposition and argue that there are several advantages of exploratory hypothesis tests that can make their results more compelling than those of confirmatory hypothesis tests. We also consider some potential disadvantages of exploratory hypothesis tests and conclude that their advantages can outweigh the disadvantages. We conclude that exploratory hypothesis tests avoid researcher commitment and researcher prophecy biases, reduce the probability of data fraud, are more appropriate in the context of unplanned deviations, facilitate inference to the best explanation, and allow peer reviewers to make additional contributions at the data analysis stage. In contrast, confirmatory hypothesis tests may lead to an inappropriate level of confidence in research conclusions, less appropriate analyses in the context of unplanned deviations, and greater bias and errors in theoretical inferences.
ExploreLive feeds
About