Let’s learn to use LangChain to entry the LLM mannequin by making a chatbot utilizing OpenAI and Llama LLM modes.
For background, first let’s have an thought about LLMs, LangChain, and fastAPI.
Giant Language Fashions (LLM) are AI fashions skilled on large chunks of information that may perceive our enter and generate a passable response to the given enter. Specifically, LLMs are Neural Community known as transformers.
LangChain is a framework that permits us to develop functions offered by LLMs. It simplifies the appliance life cycle from manufacturing to deployment.
FastAPI is a quick, high-performance internet framework for API improvement in Python. It goals for speedy improvement and maximize developer productiveness by options like computerized validation, and documentation.
In the present day, we shall be utilizing OpenAI Turbo 3.5 and open-source Llama3 fashions.
Step 1: Let’s begin by getting API keys. Check in to OpenAI here and get the API key for starters, OpenAI offers 5 {dollars} credit score totally free however it doesn’t work effectively. Additionally, register to Langchain here and get API keys. Then retailer the APIs in ‘.env’.
LANGCHAIN_API_KEY= "lsv2_pt_d186f1a84fccab6117_b530a19b63"
OPENAI_API_KEY= "sk-non1-yjPTlT3BlbkFJbrPfWNOW1qz8fXYU"
LANGCHAIN_PROJECT="CHATBOT"
Step 2: For Llama, obtain and set up Ollama kind here and run ‘ollama run llama3’ command in terminal. You should utilize different fashions to as your want.
Step 3: Create a Python surroundings and set up dependencies. First, create a necessities.txt file and record the next libraries.
- langchain
- langchain-community
- langchain-openai
- langserve
- python-dotenv
- streamlit
- uvicorn
Then, run the next instructions within the terminal.
python -m venv venv //for creating surroundings
venvScriptsactivate //for activating surroundings
pip set up necessities.txt //for putting in dependencies
Step 4: Create a python file in my case “app.py”. On this file we are going to study to create API utilizing FastAPI for LLMs.
- import dependencies
from fastapi import FastAPI
from langchain.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_community.llms import Ollama
from dotenv import load_dotenv
from langserve import add_routes
import uvicorn
import os
2. Load the API keys saved in .env. Do not forget that it is very important retailer API keys in .env because it ensures privateness.
load_dotenv()
os.environ["LANGCHAIN_TRACING_V2"]="true"
os.environ["LANGCHAIN_API_KEY"]=os.getenv("LANGCHAIN_API_KEY")
3. Initialize the FastAPI and fashions to make use of on this case OpenAI gpt-3.5-turbo and Llama3.
app=FastAPI(
title="Python Assistent",
model="1.0",
decsription="API Server"
)
mannequin=ChatOpenAI(mannequin="gpt-3.5-turbo")
llm=Ollama(mannequin="llama3")
4. Create an preliminary immediate template for each fashions. Immediate templates convert uncooked consumer enter to higher enter to the LLM. A Immediate Template is used to outline and handle the prompts that information language mannequin interactions. These templates permit for dynamic and reusable immediate creation by enabling the insertion of variables into predefined textual content constructions.
prompt1=ChatPromptTemplate.from_template("Write me an essay about {subject} with 100 phrases")
prompt2=ChatPromptTemplate.from_template("Assist to put in writing a python code for offered {subject}.")
5. add routes for every mannequin
add_routes(
app,
prompt1|mannequin,
path="/essay"
)
add_routes(
app,
prompt2|llm,
path="/code"
)
if __name__=="__main__":
uvicorn.run(app,host="localhost",port=8000)
Now, you may merely run this system you get an identical interface in your native host(browser) as in Determine 1. Merely after including in ‘/docs’, you may see all the main points of the API created known as Swagger UI documentation as in Determine 2. Hurray!! API has been created.
Step 5: Now, create one other Python file in my case “shopper.py”. Now because the title suggests on this step, we are going to learn to work together with the API that has been created. We are going to create an app utilizing Streamlit that can work together with API.
- Import dependencies and outline the features to name APIs.
import requests
import streamlit as stdef get_openai_response(input_text):
response=requests.put up("http://localhost:8000/essay/invoke",
json={'enter':{'subject':input_text}})
return response.json()['output']['content']
def get_ollama_response(input_text):
response=requests.put up(
"http://localhost:8000/code/invoke",
json={'enter':{'subject':input_text}})
return response.json()['output']['content']
Right here within the ‘get_openai_response’ and ‘get_ollama_response’ features ‘request.put up’ is used to name the API that’s working in our native host. We move the URL of the native host plus the trail of the mannequin that we created whereas creating the API as a parameter of features. Additionally, we move JSON as enter. Right here ‘subject’ is used as a result of we used it whereas making a immediate template.
2. Now let’s initialize the Streamlit framework and run it in seperate terminal utilizing command ‘streamlit run file_name’
st.title('Langchain Demo With LLAMA3 API')
input_text=st.text_input("Write an essay on(utilizing openAI)")
input_text1=st.text_input("Write code (utilizing llama3)")if input_text:
st.write(get_openai_response(input_text))
if input_text1:
st.write(get_ollama_response(input_text1))
Hurray, we now have created API utilizing FastAPI for LLMs and learnt to make use of each open-source and paid LLM fashions
