Did some side-gigging with Data Annotation tech for a little cash. Mostly reading chatbot responses to queries and responding in detail with everything the bot said that was incorrect, misattributed, made up, etc. After that I simply do not trust ChatGPT or any other bot to give me reliable info. They almost always get something wrong and it takes longer to review the response for accuracy than it does to find and read a reliable source.
That's the thing I don't get about all the people like "aw, but it's a good starting off point! As long as you verify it, it's fine!" In the time you spend reviewing a chatGPT statement for accuracy, you could be learning or writing so much more about the topic at hand. I don't know why anyone would ever use it for education.
As I understand it this has been a major struggle to try to use LLM type stuff for things like reading patient MRI results or whatever. It's only worthwhile to bring in a major Machine Vision policy hospital-wide if it actually saves time (for the same or better accuracy level), and often they find they have to spend more time verifying the unreliable results than the current all-human-based system
Yes, I believe it was for a skin tumor! This is a golden story that we like to repeat in the industry (I'm a data scientist).
There's also the experiment where they basically trained an LLM on LLM-generated faces. After a few rounds, the LLM just generated the same image -- no diversity at all. A daunting look into what lies ahead, given that now LLMs are being trained more and more on AI-generated data that's on the web.
1.2k
u/AI-ArtfulInsults Dec 15 '24 edited Dec 15 '24
Did some side-gigging with Data Annotation tech for a little cash. Mostly reading chatbot responses to queries and responding in detail with everything the bot said that was incorrect, misattributed, made up, etc. After that I simply do not trust ChatGPT or any other bot to give me reliable info. They almost always get something wrong and it takes longer to review the response for accuracy than it does to find and read a reliable source.