Class TrajectoryEvalChain

A chain for evaluating ReAct style agents.

This chain is used to evaluate ReAct style agents by reasoning about the sequence of actions taken and their outcomes.

Hierarchy

AgentTrajectoryEvaluator
- TrajectoryEvalChain

Constructors

constructor

new TrajectoryEvalChain(fields): TrajectoryEvalChain
Parameters
- fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>
Returns TrajectoryEvalChain
Inherited from AgentTrajectoryEvaluator.constructor
- Defined in langchain/src/chains/llm_chain.ts:118

Properties

llm

llm: BaseLanguageModelInterface

outputKey

outputKey: string = "text"

outputParser

outputParser: TrajectoryOutputParser = ...

prompt

prompt: BasePromptTemplate

requiresInput

requiresInput: boolean = true

requiresReference

requiresReference: boolean = false

`Optional` criterionName

criterionName?: string

`Optional` evaluationName

evaluationName?: string = ...

`Optional` llmKwargs

llmKwargs?: any

`Optional` memory

memory?: any

`Optional` skipInputWarning

skipInputWarning?: string = ...

`Optional` skipReferenceWarning

skipReferenceWarning?: string = ...

Accessors

inputKeys

get inputKeys(): any
Returns any
Inherited from AgentTrajectoryEvaluator.inputKeys
- Defined in langchain/src/chains/llm_chain.ts:110

outputKeys

get outputKeys(): string[]
Returns string[]
Inherited from AgentTrajectoryEvaluator.outputKeys
- Defined in langchain/src/chains/llm_chain.ts:114

Methods

apply

apply(inputs, config?): Promise<ChainValues[]>
Parameters
- inputs: ChainValues[]
- Optional config: any[]
Returns Promise<ChainValues[]>

⚠️ Deprecated ⚠️

Use .batch() instead. Will be removed in 0.2.0.

This feature is deprecated and will be removed in the future.

It is not recommended for use.

Call the chain on all inputs in the list
Inherited from AgentTrajectoryEvaluator.apply
- Defined in langchain/src/chains/base.ts:256

call

call(values, config?): Promise<ChainValues>
Run the core logic of this chain and add to output if desired.

Wraps _call and handles memory.
Parameters
- values: any
- Optional config: any
Returns Promise<ChainValues>
Inherited from AgentTrajectoryEvaluator.call
- Defined in langchain/src/chains/llm_chain.ts:175

checkEvaluationArgs

checkEvaluationArgs(reference?, input?): void
Check if the evaluation arguments are valid.
Parameters
- Optional reference: string
  
  The reference label.
- Optional input: string
  
  The input string.
Returns void

Throws
If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.
Inherited from AgentTrajectoryEvaluator.checkEvaluationArgs
- Defined in langchain/src/evaluation/base.ts:62

evaluateAgentTrajectory

evaluateAgentTrajectory(args, callOptions?, config?): Promise<ChainValues>
Evaluate a trajectory.
Parameters
- args: LLMTrajectoryEvaluatorArgs
- Optional callOptions: unknown
- Optional config: any
Returns Promise<ChainValues>
The evaluation result.
Inherited from AgentTrajectoryEvaluator.evaluateAgentTrajectory
- Defined in langchain/src/evaluation/base.ts:351

formatReference

formatReference(reference?): string
Parameters
- Optional reference: string
Returns string
- Defined in langchain/src/evaluation/agents/trajectory.ts:183

getAgentTrajectory

getAgentTrajectory(steps): string
Get the agent trajectory as a formatted string.
Parameters
- steps: AgentStep[]
  
  The agent trajectory.
Returns string
The formatted agent trajectory.
- Defined in langchain/src/evaluation/agents/trajectory.ts:168

invoke

invoke(input, config?): Promise<ChainValues>
Invoke the chain with the provided input and returns the output.
Parameters
- input: ChainValues
  
  Input values for the chain run.
- Optional config: any
  
  Optional configuration for the Runnable.
Returns Promise<ChainValues>
Promise that resolves with the output of the chain run.
Inherited from AgentTrajectoryEvaluator.invoke
- Defined in langchain/src/chains/base.ts:86

predict

predict(values, callbackManager?): Promise<EvalOutputType>
Format prompt with values and pass to LLM
Parameters
- values: any
  
  keys to pass to prompt template
- Optional callbackManager: any
  
  CallbackManager to use
Returns Promise<EvalOutputType>
Completion from LLM.
Example
```
llm.predict({ adjective: "funny" })
```
Inherited from AgentTrajectoryEvaluator.predict
- Defined in langchain/src/chains/llm_chain.ts:241

prepOutputs

prepOutputs(inputs, outputs, returnOnlyOutputs?): Promise<Record<string, unknown>>
Parameters
- inputs: Record<string, unknown>
- outputs: Record<string, unknown>
- returnOnlyOutputs: boolean = false
Returns Promise<Record<string, unknown>>
Inherited from AgentTrajectoryEvaluator.prepOutputs
- Defined in langchain/src/chains/base.ts:148

run

run(input, config?): Promise<string>
Parameters
- input: any
- Optional config: any
Returns Promise<string>

Deprecated
Use .invoke() instead. Will be removed in 0.2.0.
Inherited from AgentTrajectoryEvaluator.run
- Defined in langchain/src/chains/base.ts:188

serialize

serialize(): SerializedLLMChain
Returns SerializedLLMChain

Deprecated
Inherited from AgentTrajectoryEvaluator.serialize
- Defined in langchain/src/chains/llm_chain.ts:269

`Static` deserialize

deserialize(data): Promise<LLMChain<string, any>>
Parameters
- data: SerializedLLMChain
Returns Promise<LLMChain<string, any>>
Inherited from AgentTrajectoryEvaluator.deserialize
- Defined in langchain/src/chains/llm_chain.ts:253

`Static` fromLLM

fromLLM(llm, agentTools?, chainOptions?): Promise<TrajectoryEvalChain>
Create a new TrajectoryEvalChain.
Parameters
- llm: BaseChatModel
- Optional agentTools: StructuredToolInterface[]
  
  The tools used by the agent.
- Optional chainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>, "llm">>
  
  The options for the chain.
Returns Promise<TrajectoryEvalChain>
- Defined in langchain/src/evaluation/agents/trajectory.ts:130

`Static` resolveTrajectoryPrompt

resolveTrajectoryPrompt(prompt?, agentTools?): any
Parameters
- Optional prompt: any
- Optional agentTools: StructuredToolInterface[]
Returns any
- Defined in langchain/src/evaluation/agents/trajectory.ts:94

`Static` toolsDescription

toolsDescription(agentTools): string
Get the description of the agent tools.
Parameters
- agentTools: StructuredToolInterface[]
Returns string
The description of the agent tools.
- Defined in langchain/src/evaluation/agents/trajectory.ts:115

Class TrajectoryEvalChain

Hierarchy

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>

Returns TrajectoryEvalChain

Properties

llm

outputKey

outputParser

prompt

requiresInput

requiresReference

Optional criterionName

Optional evaluationName

Optional llmKwargs

Optional memory

Optional skipInputWarning

Optional skipReferenceWarning

Accessors

inputKeys

Returns any

outputKeys

Returns string[]

Methods

apply

Parameters

inputs: ChainValues[]

Optional config: any[]

Returns Promise<ChainValues[]>

⚠️ Deprecated ⚠️

call

Parameters

values: any

Optional config: any

Returns Promise<ChainValues>

checkEvaluationArgs

Parameters

Optional reference: string

Optional input: string

Returns void

Throws

evaluateAgentTrajectory

Parameters

args: LLMTrajectoryEvaluatorArgs

Optional callOptions: unknown

Optional config: any

Returns Promise<ChainValues>

formatReference

Parameters

Optional reference: string

Returns string

getAgentTrajectory

Parameters

steps: AgentStep[]

Returns string

invoke

Parameters

input: ChainValues

Optional config: any

Returns Promise<ChainValues>

predict

Parameters

values: any

Optional callbackManager: any

Returns Promise<EvalOutputType>

Example

prepOutputs

Parameters

inputs: Record<string, unknown>

outputs: Record<string, unknown>

returnOnlyOutputs: boolean = false

Returns Promise<Record<string, unknown>>

run

`Optional` criterionName

`Optional` evaluationName

`Optional` llmKwargs

`Optional` memory

`Optional` skipInputWarning

`Optional` skipReferenceWarning

`Optional` config: any[]

`Optional` config: any

`Optional` reference: string

`Optional` input: string

`Optional` callOptions: unknown

`Optional` config: any

`Optional` reference: string

`Optional` config: any

`Optional` callbackManager: any

`Optional` config: any

`Static` deserialize

`Static` fromLLM

`Optional` agentTools: StructuredToolInterface[]

`Optional` chainOptions: Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>, "llm">>

`Static` resolveTrajectoryPrompt

`Optional` prompt: any

`Optional` agentTools: StructuredToolInterface[]

`Static` toolsDescription