Member-only story
Understanding PandasAI
The simplicity of using an LLM for data understanding
It seems that finding novel uses for LLMs is one of our community’s hobbies. Pandas-AI is a Python library that integrates generative artificial intelligence capabilities into Pandas, making data frames conversational. I decided to understand how someone can manipulate any LLM in order to ask data questions.
I decided to look at the backend, and I found out that the code simply converses with the LLM, then it creates Python code from your data question, runs it, corrects the code if it fails, and repeats until there is an answer to your data question.
This mechanism is quite interesting as it uses an aggregation of concepts that were discovered early on, as a way to ask data, or actual business questions and get answers.
The concept can be generalized to other solutions, but the gist of it is quite simple
1️⃣ 𝚛𝚞𝚗()->𝚐𝚎𝚗𝚎𝚛𝚊𝚝𝚎_𝚌𝚘𝚍𝚎(𝚌𝚘𝚗𝚟𝚎𝚛𝚜𝚊𝚝𝚒𝚘𝚗)->𝚛𝚞𝚗_𝚌𝚘𝚍𝚎()->𝚐𝚎𝚗𝚎𝚛𝚊𝚝𝚎_𝚌𝚘𝚍𝚎() + 𝚎𝚛𝚛𝚘𝚛_𝚌𝚘𝚛𝚛𝚎𝚌𝚝𝚒𝚗𝚐_𝚒𝚗𝚜𝚝𝚛𝚞𝚌𝚝𝚒𝚘𝚗
2️⃣ 𝚕𝚘𝚘𝚙 𝚞𝚗𝚝𝚒𝚕 𝚌𝚘𝚖𝚙𝚕𝚎𝚝𝚒𝚘𝚗 𝚘𝚛 𝟹 𝚛𝚎𝚝𝚛𝚒𝚎𝚜
you can see how it works here
Dr. Ori Cohen has a Ph.D. in Computer Science with a focus on machine learning. He is a Senior Director of Data and the author of the ML & DL Compendium and StateOfMLOps.com.
Substack | LinkedIn| Medium | OriCohen.com | book.mlcompendium.com
