The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Pythonbb
| Jan 23, 2024
0  |  Read Time 0 min
link
Publish Date
Number
reflection
abstract
GPT-3 and related models are fragile under identifier swaps in programs, suggesting that these models may only possess a shallow understanding of code.
Status
Type
evaluation
Author
  • Valine
Catalog