Published on
by 
James Ravenscroft
Fair warning: Some of the quotes in this post are a bit sweary.
I've been reflecting on a few articles that I've read over the last couple of days about consumer trends in devices and parallels with generated content. I'm sure everyone's sick of "yet another hot take on enshittification" - but what about an optimistic one?
Published on
by James Ravenscroft
Gary’s article dropped on Friday and has been widely circulated and commented upon over the weekend.
It shows that LLMs struggle to generalise outside of their prompt (they know that Tom Cruise’s mum is Mary Lee Pfeifer but don’t know that Mary Lee Pfeiffer’s son is Tom Cruise - but there are many more examples). This is a known weakness of neural networks that I wrote about in my EACL2021 paper and that has previously been documented as far back as the 90s. What’s interesting is that it still holds today for these massive models with billions of parameters.
Read more...
Published on
by James Ravenscroft
Typesetting blocks form alphabet spaghetti a bit like a language model might spit out. Photo by Raphael Schaller on Unsplash
There is sooo much hype around LLMs at the moment. As an NLP practitioner of 10 years (I built Partridge 1 in 2013), it’s exhausting and quite annoying and amongst the junior ranks, there’s a lot of despondency and dejection and a feeling of “what’s the point? ClosedOpenAI have solved NLP”.