MLflow
MLflow is a versatile, expandable, open-source platform for managing workflows and artifacts across the machine learning lifecycle. It has built-in integrations with many popular ML libraries, but can be used with any library, algorithm, or deployment tool. It is designed to be extensible, so you can write plugins to support new workflows, libraries, and tools.
This notebook goes over how to track your LangChain experiments into your MLflow Server
External examplesβ
MLflow
provides several examples for the LangChain
integration:
Exampleβ
%pip install --upgrade --quiet azureml-mlflow
%pip install --upgrade --quiet pandas
%pip install --upgrade --quiet textstat
%pip install --upgrade --quiet spacy
%pip install --upgrade --quiet langchain-openai
%pip install --upgrade --quiet google-search-results
!python -m spacy download en_core_web_sm
import os
os.environ["MLFLOW_TRACKING_URI"] = ""
os.environ["OPENAI_API_KEY"] = ""
os.environ["SERPAPI_API_KEY"] = ""
from langchain.callbacks import MlflowCallbackHandler
from langchain_openai import OpenAI
API Reference:
"""Main function.
This function is used to try the callback handler.
Scenarios:
1. OpenAI LLM
2. Chain with multiple SubChains on multiple generations
3. Agent with Tools
"""
mlflow_callback = MlflowCallbackHandler()
llm = OpenAI(
model_name="gpt-3.5-turbo", temperature=0, callbacks=[mlflow_callback], verbose=True
)
# SCENARIO 1 - LLM
llm_result = llm.generate(["Tell me a joke"])
mlflow_callback.flush_tracker(llm)
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
API Reference:
# SCENARIO 2 - Chain
template = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title.
Title: {title}
Playwright: This is a synopsis for the above play:"""
prompt_template = PromptTemplate(input_variables=["title"], template=template)
synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=[mlflow_callback])
test_prompts = [
{
"title": "documentary about good video games that push the boundary of game design"
},
]
synopsis_chain.apply(test_prompts)
mlflow_callback.flush_tracker(synopsis_chain)
from langchain.agents import AgentType, initialize_agent, load_tools
API Reference:
# SCENARIO 3 - Agent with Tools
tools = load_tools(["serpapi", "llm-math"], llm=llm, callbacks=[mlflow_callback])
agent = initialize_agent(
tools,
llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
callbacks=[mlflow_callback],
verbose=True,
)
agent.run(
"Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?"
)
mlflow_callback.flush_tracker(agent, finish=True)