This is a setup for running llms locally inside of a docker container and behind FastAPI. The API supports both full generation and streaming endpoints to hit. The ollama-template folder is where you will find the FastAPI code as well as the Docker setup to get Ollama and the API up and running. The ollama-test folder is a simple Docker container that you can spin up and run `main.py` in to test both the `/stream` endpoint and `/generate` endpoint of the API in the other container. These containers use a Docker network bridge to gain access to interact. To find the code and further instructions on how to utilize the code please click on the code link above. Enjoy!

‍