< cd ../gallery
[AI & ML]
LLMWebCrawler
A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.
@Aavache
maintainer
★ 103 stars
# README
A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG. Maintained by Aavache on GitHub, where it has earned 103 stars from the community.
It's actively developed around api, distributed-computing, fastapi, and is a solid reference for anyone building with these tools.
# tags
# install
npm install LLMWebCrawlermore FastAPI repos
