Changing your phone provider = changing your atmosphere provider → keep your phone number = keep your identity (username) Changing your phone = changing what app you use* → keep your contacts & photos = keep your followers, following, posts, etc. You can change providers without changing phones.

And why, while flawed, the study isn't meaningless or worthless - everyone's take is just lacking nuance:

tbh even these takes are too extreme though. see my thread on what the paper actually says:

👀

and it can be another color if you use a different color scheme omg love it.

oxford common stay winning

It’s just atproto but European and kinda weird suss

True - I also think this probably enables a much more diverse range of input from a more diverse group of people than you could get through more “traditional” means

prior AI usage. That is, how often and in what situations they have used AI previously outside the study. Especially since it’s more popular in everyday life, seems like an important consideration.

(or, I’ll add, how AI usage itself might change over time) - how long these effects last aren’t known; these people were tested within a 15 min window. testing with AI in morning and without in evening could change the results. One thing they didn’t note or do is like to see too is assess

Yes mostly. The paper (still a preprint, so not peer-reviewed) does start off kinda strong in its claims compared to what the results show. But they acknowledge all the limitations too: - this captured brief AI exposure, so did not look at how these effects change with usage over time

- excluded results if person didn’t pass attention check or didn’t answer pretest correctly If they did the same survey after experiment 3 that wasn’t reported which is odd (tho could be they did Exp 3 before Exp 2, papers are weird that way)

Experiment 3: 201 people, reading comprehension test instead of fractions. - same pretest, learning, test setup as experiment 2 - pretest = 1 question - AI grp: 5 questions with AI, 3 without - control = 8 questions - overrode test answer with “skipped” if question answered in <5 seconds

With experiment 2 they also, afterward collected: - multiple-choice answers to how AI assistance was used Choices: - didn’t use AI (checked against logs to confirm) - used for hints/clarification - asked for solution **persistence effect only seen in those who asked for a direct solution**

For experiment 2 “…the skip rate difference did not reach significance…possibly due to heterogeneity in how participants used the AI. We examine these patterns in the next [experiment].”

- AI group sidebar was the chat window, control group sidebar was the pretest problem + solution - Test = 3 problems with no sidebar - Excluded from results if person didn’t pass attention check (e.g. “choose 10” and don’t choose 10) - Excluded from results if didn’t solve 1/3 pretest

Continuing now that I'm not running around like headless chicken: Experiment 2 repeated Experiment 1 with these changes: - 667 people - There was a pretest, learning, and test phase - Pretest = 3 one step problems, shown answer after - Learning = 11 problems with a sidebar

Group 1: AI = told access to AI for some problems, encouraged to use however they want. Had AI first 12 problems then no AI last 3. “choosing to skip reflects a deliberate decision not to engage, making it a clear measure of motivation and persistence, independent of ability.” Group 2: No AI.

Experiment 1 = 354 participants, randomly assigned to solve “15 fraction problems…of varying difficulty”. No penalty for wrong answers, payment not dependent on how many they got correct. If wrong, answer shown. Had option to Skip questions.