Thoughts

I have very mixed feelings about #ChatGPT and related large language models (LLMs). They are certainly a cool tool, but I also currently don’t believe that these “stochastic parrots” will easily replace search, article/paper writing, coding, and similar tasks. That is, they can be used to augment, but not replace those human activities - at least in the near future.

One more specific example is #Github #Copilot: I am impressed by the code suggested with small “query blocks”, either in the form of method signatures or even just comments. While I wouldn’t ever just Tab-complete that code into a production codebase (and I hope others also shy away from it), using it as a smarter auto-complete can be a total time-saver. While still learning #Rust, I find it a great tool to provide me with correct syntax while I try to express the semantic concepts in my head. That is, the #VSCode + #Copilot + #rustanalyzer based workflow is a magnitude faster than what I can achieve, at my current level of competence with #Rust, with any other IDE. However, on the conceptual level, its suggestions range from spot-on (supposedly for parts that have been used in the same form in many Github projects already) to “terrible, awful, don’t even think about it”. Don’t trust the concepts generated by #Copilot, but for getting a concept into correct syntax, it can be a great help.

I find the same to be true of #ChatGPT: it produces correct (human language) syntax, but its concepts on a semantic level are more random than anything else. It doesn’t have a reasonable world model, and without such a world model, even internal consistency of statements (let alone factual validity in alignment with what we consider the real world) is darn hard (impossible?) to achieve. It’s currently a toy, and should only be used for toy purposes. On that note, the best use I could personally find for #ChatGPT is to reply to spammers, who operate on a comparable level of factual and consistent concepts (i.e., #BS): https://github.com/rmayr/chatgpt-email

Maybe #ReinforcementLearning will at some point allow LLMs to be factually more accurate, including the ability to cite sources. I don’t know, as I’m not an expert in this field. But the current #UnsupervisedLearning approaches with just scraping massive amounts of (syntax, but not semantic) web content seem fundamentally flawed to me.

On a sidenote, if the #OpenAI mission is really to “ensure that artificial general intelligence benefits all of humanity”, shouldn’t the trained models be, erm, #open (source)? After all, the training data is publicly generated, so it seems more beneficial to humanity if the derivations are also #open for building upon them.