Rabbit Hole Crawler · Bruno Moura

Problem: Understanding what a channel covers means watching its videos or reading the titles one by one.
Approach: Collects the titles with yt-dlp (10 by default), classifies the topic of each one with a local LLM (Mistral via Ollama) and points out similar topics with embeddings and FAISS. The result is written to JSON.
Status: Study. A project built to learn and try out a technique.

data/output.json

1 Collection yt-dlp

titulo

ELE SÓ IA VISITAR O PAI NA FAZENDA, MAS ACABOU SENDO ABDUZIDO POR ALIENÍGENAS!
2 Classification Mistral via Ollama

tema

Ficção Científica ou Fantasia
3 Similarity MiniLM + FAISS

temas_semelhantes
- Televisão (TV) ou Cinema
- Aventura marinha ou Animais Marinhos

Fig. 1 Output record published in the project’s README, in Portuguese as published, annotated with the pipeline step that produces each field.

How it works

Collection. yt-dlp reads the channel’s metadata without downloading videos (extract_flat) and returns the first titles, 10 by default.
Classification. Each title becomes a few-shot prompt sent to Mistral, run locally by Ollama, which answers with the topic in one or two words.
Similarity. The topics are turned into embeddings with all-MiniLM-L6-v2 and indexed in FAISS (IndexFlatL2). For each topic, the index returns its nearest neighbors.
Output. Title, topic and similar topics are written to data/output.json.

Sample output

Sample record published in the project’s README (in Portuguese, as published):

{
  "titulo": "ELE SÓ IA VISITAR O PAI NA FAZENDA, MAS ACABOU SENDO ABDUZIDO POR ALIENÍGENAS!",
  "tema": "Ficção Científica ou Fantasia",
  "temas_semelhantes": ["Televisão (TV) ou Cinema", "Aventura marinha ou Animais Marinhos"]
}

Current state

A study project for educational purposes: a short pipeline to experiment with local AI, NLP and semantic similarity.