C Transformers
The C Transformers library provides Python bindings for GGML models.
This example goes over how to use LangChain to interact with C Transformers
models.
Install
%pip install --upgrade --quiet ctransformers
Load Model
from langchain_community.llms import CTransformers
llm = CTransformers(model="marella/gpt-2-ggml")
API Reference:
Generate Text
print(llm.invoke("AI is going to"))
Streaming
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
llm = CTransformers(
model="marella/gpt-2-ggml", callbacks=[StreamingStdOutCallbackHandler()]
)
response = llm.invoke("AI is going to")
API Reference:
LLMChain
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
template = """Question: {question}
Answer:"""
prompt = PromptTemplate.from_template(template)
llm_chain = LLMChain(prompt=prompt, llm=llm)
response = llm_chain.run("What is AI?")