Experimental browser for the Atmosphere
We find that LLMs' metalinguistic judgments are inferior to direct probability-based comparisons, suggesting that negative results relying on metalinguistic prompts cannot be taken as conclusive evidence that an LLM lacks a particular linguistic generalization. 3/5
Oct 24, 2023, 3:04 PM
{ "uri": "at://did:plc:t7jbj4w3uo2sus4bc3znsspa/app.bsky.feed.post/3kciyose4ru23", "cid": "bafyreiffvzcm2blu2wwraixiqu42q6ufiiersjcyxkv7py6kmdyyuyr6di", "value": { "text": "We find that LLMs' metalinguistic judgments are inferior to direct probability-based comparisons, suggesting that negative results relying on metalinguistic prompts cannot be taken as conclusive evidence that an LLM lacks a particular linguistic generalization. 3/5", "$type": "app.bsky.feed.post", "langs": [ "en" ], "reply": { "root": { "cid": "bafyreiclw56tvjjsi4r5tv53erbkmmzwjf4ohc2rttnowg6tivm4k7cdfi", "uri": "at://did:plc:t7jbj4w3uo2sus4bc3znsspa/app.bsky.feed.post/3kciylu4bpi2w" }, "parent": { "cid": "bafyreicuiel4d4uun335i45n55teobvcavcx2vymklzdyh6vkwwwvzmale", "uri": "at://did:plc:t7jbj4w3uo2sus4bc3znsspa/app.bsky.feed.post/3kciynwptmj2w" } }, "createdAt": "2023-10-24T15:04:58.266Z" } }