It’s bizarre to me that there are still people claiming that LLMs are not reasoning or are not creative when by any objective measure they are obviously creative reasoners! By objective measure I mean a test that evaluates creativity and reasoning by evaluating outputs not by idle philosophical speculation that rules AIs out by definition. Here’s a good paper, Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers, which illustrates one such test. The authors asked top researchers in the field of natural language processing to propose research ideas which were then presented in a standardized format to a ratings panel of other NLP experts. The AI created ideas were judged more creative than the human ideas.
Now one might argue that the humans weren’t giving their best ideas–some data in the paper suggests they were giving ideas at the median of those for top researchers–and humans might also be looking for ideas that were perhaps easier to get funding precisely because they were less creative but more doable. Either way, however, the AIs are coming up with good ideas that could usefully supplement human generated ideas.
The post LLMs are Creative Reasoners appeared first on Marginal REVOLUTION.