• superkret@feddit.de
    link
    fedilink
    English
    arrow-up
    74
    ·
    edit-2
    1 year ago

    online study
    not peer reviewed
    “published” on arxiv (which is a public document server, not a journal)
    study and authors not named or linked in the article

    tl/dr: “Someone uploaded a pdf and we’re writing about it.”

    • barsoap@lemm.ee
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      1 year ago

      I suppose it’s this paper. Most prolific author seems to be Gene Tsudik, h-index of 103. Yeah that’s not “someone”. Also the paper is accepted for USENIX Security 2023, which is actually ongoing right now.

      Also CS doesn’t really do academia like other sciences, being somewhere on the intersection of maths, engineering, and tinkering. Shit’s definitely not invalid just because it hasn’t been submitted to a journal this could’ve been a blog post but there’s academics involved so publish or perish applies.

      Or, differently put: If you want to review it, bloody hell do it it’s open access. A quick skim tells me “way more thorough than I care to read for the quite less than extraordinary claim”.

    • CookieJarObserver@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      9
      arrow-down
      1
      ·
      1 year ago

      I mean its pretty obvious that nowadays AI is absolutely capable of doing that and some people are just blind or fat finger the keyboard.

    • Zeth0s@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      8
      ·
      edit-2
      1 year ago

      You are overrating peer reviewing. It’s basically a tool to help editors to understand if a paper “sells”, to improve readability and to discard clear garbage.

      If methodologies are not extremely flawed, peer reviewing almost never impact quality of the results, as reviewers do not redo the work. From the “trustworthy” point of view, peer reviewing is comparable to a biased rng. Google for actual reproducibility of published experiments and peer-reviewing biases for more details

      Preprints are fine, just less polished

        • Zeth0s@lemmy.world
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          5
          ·
          edit-2
          1 year ago

          Unfortunately not. https://www.nature.com/articles/533452a

          Most peer reviewed papers are non reproducible. Peer review has the primary purpose of telling the editor how sellable is a paper in a small community he only superficially knows, and to make it more attractive to that community by suggesting rephrasing of paragraphs, additional references, additional supporting experiment to clarify unclear points.

          But it doesn’t guarantees methodology is not flawed. Editor chooses reviewer very superficially, and reviews are mainly driven by biases, and reviewers cannot judge the quality of a research because they do not reproduce it.

          Honesty of researchers is what guarantees quality of a paper

          • C4d@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            1
            ·
            1 year ago

            Yes. A senior colleague sometimes tongue-in-cheek referred to it as Pee Review.

            • Zeth0s@lemmy.world
              link
              fedilink
              English
              arrow-up
              3
              arrow-down
              2
              ·
              edit-2
              1 year ago

              The downvotes to my comments shows that no many people here has ever done research or knows the editorial system of scientific journals :D

              • C4d@lemmy.world
                link
                fedilink
                English
                arrow-up
                2
                ·
                1 year ago

                There is some variation across disciplines; I do think that in general the process does catch a lot of frank rubbish (and discourages submission of obvious rubbish), but from time to time I do come across inherently flawed work in so-called “high impact factor” and allegedly “prestigious” journals.

                In the end, even after peer review, you need to have a good understanding of the field and to have developed and applied your critical appraisal skills.

                • barsoap@lemm.ee
                  link
                  fedilink
                  English
                  arrow-up
                  5
                  ·
                  1 year ago

                  And TBF just getting on arxiv also means you jumped a bullshit hurdle: Roughly speaking you need to be in a position in academia, or someone there needs to vouch for the publication. At the same time getting something published there isn’t exactly prestigious so there’s no real incentive to game the system, as such the bar is quite low but consistent.

                  • Zeth0s@lemmy.world
                    link
                    fedilink
                    English
                    arrow-up
                    3
                    ·
                    edit-2
                    1 year ago

                    Arxiv is a pre print archive. Many very prestigious researchers put their pre prints there. It is as credible as any journal (more than many out there nowadays). Its presentation is just less curated and a selection is missing, because there is no editor. Readers of a paper must know what they are reading, and must critically assess it.

                • Zeth0s@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  ·
                  edit-2
                  1 year ago

                  Absolutely. One needs to know what is reading. That’s why pre prints are fine.

                  High impact factor journals are full of works purposely wrong, made because author wants the results that readers are looking for (that is the easiest way to be published in high impact factor journal).

                  https://www.timeshighereducation.com/news/papers-high-impact-journals-have-more-statistical-errors

                  It’s the game. Reader must know how to navigate the game. Both for peer reviewed papers and pre prints