Member-only story
Understanding Scikit-LLM
Using LLMs as black box classifiers
I was curious to understand how Scikit-LLM works in practice. According to the repo’s author, Scikit-LLM integrates powerful language models like ChatGPT into Scikit-learn for enhanced text analysis tasks.
Looking at the backend, it seems that it simply wraps zero-shot prompts for multi-class and multi-label classes. Let’s break down what it does in practice.
1️⃣ It asks GPT to act as a classifier by inlining the labels and the text asking for a JSON response.
2️⃣ Defaulting to gpt3.5-turbo and ada-002 embeddings, but flexible in using others.
3️⃣ Returning prediction & probabilities.
If you are curious to learn more, you can see the code for text prompts here and the code for classifier classes here
On a personal note, given the volatility and nondeterministic nature of prompt engineering, and that this isn’t a traditional “algorithm”, I suspect that this will not be introduced into Scikit-learn as one of the internal classes any time soon.
What do you think?
Dr. Ori Cohen has a Ph.D. in Computer Science with a focus on machine learning. He is a Senior Director of Data and the author of the ML & DL Compendium and StateOfMLOps.com.
Substack | LinkedIn| Medium | OriCohen.com | book.mlcompendium.com
