@lhdjung.bsky.social
PhD student in psychological metascience at Uni Bern | R package developer – {scrutiny}, {unsum}, and more | https://github.com/lhdjung | https://lhdjung.com/ | forensic metascience | error detection | missing value handling | #rstats
@lhdjung.bsky.socialI looked into the thread for precisely this.
@lhdjung.bsky.socialMany new forensic meta-science tools aim to detect trustworthiness problems in published research at scale.
Yet how trustworthy are these tools themselves? We – @ianhussey.mmmdata.io, @jamiecummins.bsky.social, and I – have some concerns.
@ianhussey.mmmdata.ioReposted by @lhdjung.bsky.socialOur reflections on the recent "AI and Research Integrity" meeting in Montreal:
We are somewhat worried that many AI trustworthiness assessment tools are not trustworthy.
@lhdjung.bsky.social
@jamiecummins.bsky.social
trustworthy.scientific.claims/posts/trustw...
@jamiecummins.bsky.socialReposted by @lhdjung.bsky.socialI sincerely hope this paper will be seen as a turning point in how the medical sciences think and approach the issue of trustworthiness assessment (and I hope the social sciences follow suit!).
Congrats to @jdwilko.bsky.social and colleagues (incl. @ianhussey.mmmdata.io and @lhdjung.bsky.social)!
@sanjaysrivastava.comReposted by @lhdjung.bsky.socialFully expecting this to be based on a tiny self-selected sample predisposed to aggression, with participants coached on how to act, exaggerated performances for observers, all run by a host with a flair for showmanship but questionable ethical instincts. You know, a direct replication
@lhdjung.bsky.socialIf you want more on programming as opposed to analysis scripts, there is also Advanced R (adv-r.hadley.nz/index.html). Chapters 3-5 or maybe 3-6 explain some of the deeper ideas of how R code behaves. The rest of the book is also great but can go way into the weeds.
@lhdjung.bsky.socialNot a terribly original reply but it can be helpful to read specific chapters of R for Data Science (r4ds.hadley.nz) on topics you feel unsure about, even if you've read the book before. When I just started out with R, I skipped parts of it that didn't click, but later they made perfect sense to me.
@jamiecummins.bsky.socialReposted by @lhdjung.bsky.socialMachine learning and generative LLMs are subject to many of the same drawbacks and issues as general empirical research in the typical publishing system. In this piece now published at NHB we argue that, like for empirical research, ML and LLMs can benefit from the use of Registered Reports.