Skip to content

Latest commit

 

History

History
19 lines (15 loc) · 1.08 KB

readme.md

File metadata and controls

19 lines (15 loc) · 1.08 KB
Raw

Simple LLM Based Search

Setup Ollama and Models

  1. Install Ollama

  2. Download the model - we are getting tinyllama which is a very compact model. And it should run on most machines.

    • ollama pull tinyllama:latest

  3. Download model for creating embeddings. The all-minilm aims to train sentence embedding models on very large sentence level datasets using a self-supervised contrastive learning objective.

    • ollama pull all-minilm:latest

  4. Start, and make sure ollama is available at http://localhost:11434/

  5. Install the packages in requirements.ttx

  6. Run indexing OLLAMA_HOST='0.0.0.0' python index_content.py

  7. Test searching QUERY="what is five plus five" USE_EMBEDDINGS=yes OLLAMA_HOST='0.0.0.0' python search.py

  8. Play around by changing the models and other parameters

  9. for a web interface - run the flask app: FLASK_HOST=0.0.0.0 FLASK_PORT=8080 python flask_server.py - then go to http://localhost:8080 on your browser.