Prediction Guard
This page covers how to use the Prediction Guard ecosystem within LangChain. It is broken into two parts: installation and setup, and then references to specific Prediction Guard wrappers.
Installation and Setupβ
- Install the Python SDK with
pip install predictionguard
- Get a Prediction Guard access token (as described here) and set it as an environment variable (
PREDICTIONGUARD_TOKEN
)
LLM Wrapperβ
There exists a Prediction Guard LLM wrapper, which you can access with
from langchain_community.llms import PredictionGuard
API Reference:
You can provide the name of the Prediction Guard model as an argument when initializing the LLM:
pgllm = PredictionGuard(model="MPT-7B-Instruct")
You can also provide your access token directly as an argument:
pgllm = PredictionGuard(model="MPT-7B-Instruct", token="<your access token>")
Finally, you can provide an "output" argument that is used to structure/ control the output of the LLM:
pgllm = PredictionGuard(model="MPT-7B-Instruct", output={"type": "boolean"})
Example usageβ
Basic usage of the controlled or guarded LLM wrapper:
import os
import predictionguard as pg
from langchain_community.llms import PredictionGuard
from langchain_core.prompts import PromptTemplate
from langchain.chains import LLMChain
# Your Prediction Guard API key. Get one at predictionguard.com
os.environ["PREDICTIONGUARD_TOKEN"] = "<your Prediction Guard access token>"
# Define a prompt template
template = """Respond to the following query based on the context.
Context: EVERY comment, DM + email suggestion has led us to this EXCITING announcement! π We have officially added TWO new candle subscription box options! π¦
Exclusive Candle Box - $80
Monthly Candle Box - $45 (NEW!)
Scent of The Month Box - $28 (NEW!)
Head to stories to get ALL the deets on each box! π BONUS: Save 50% on your first box with code 50OFF! π
Query: {query}
Result: """
prompt = PromptTemplate.from_template(template)
# With "guarding" or controlling the output of the LLM. See the
# Prediction Guard docs (https://docs.predictionguard.com) to learn how to
# control the output with integer, float, boolean, JSON, and other types and
# structures.
pgllm = PredictionGuard(model="MPT-7B-Instruct",
output={
"type": "categorical",
"categories": [
"product announcement",
"apology",
"relational"
]
})
pgllm(prompt.format(query="What kind of post is this?"))
API Reference:
Basic LLM Chaining with the Prediction Guard wrapper:
import os
from langchain_core.prompts import PromptTemplate
from langchain.chains import LLMChain
from langchain_community.llms import PredictionGuard
# Optional, add your OpenAI API Key. This is optional, as Prediction Guard allows
# you to access all the latest open access models (see https://docs.predictionguard.com)
os.environ["OPENAI_API_KEY"] = "<your OpenAI api key>"
# Your Prediction Guard API key. Get one at predictionguard.com
os.environ["PREDICTIONGUARD_TOKEN"] = "<your Prediction Guard access token>"
pgllm = PredictionGuard(model="OpenAI-gpt-3.5-turbo-instruct")
template = """Question: {question}
Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)
llm_chain = LLMChain(prompt=prompt, llm=pgllm, verbose=True)
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
llm_chain.predict(question=question)