LangSmith
LangSmith is an AI agent and LLM model evaluation platform for running evals before and after shipping, gathering expert feedback, and iterating on prompts. It supports offline and online evaluation, human annotation queues, LLM-as-judge, custom evaluators, CI/CD integration, and framework-agnostic agent evaluation.
Provider: LangChain
- active_langsmith_customers: 7K+ in AI Agent & LLM Model Evaluation Platform
- fortune_10_customers: 5 of the Fortune 10
- hosted_data_regions: GCP us-central-1 or europe-west4 in AI Agent & LLM Model Evaluation Platform
- monthly_open_source_downloads: 350M+
- self_hosting: Kubernetes cluster in AWS, GCP, or Azure; SaaS, BYOC, or self-hosted in AI Agent & LLM Model Evaluation Platform
- trains_on_customer_data: No; customer owns all rights to their data in AI Agent & LLM Model Evaluation Platform
Buying
- Public pricing
- Unknown
- Enterprise offering
- Unknown
- Founded
- Unknown
Capabilities
- Run, improve, and govern agents with LangSmith
- Build agents with or without code and choose your model
- Test agent behavior with step-by-step tracing and evals
- Deploy agents on infrastructure for long-running tasks, persistent state, and secure execution
- Monitor production with online evals, dashboards, and alerts
- Iterate through an owned feedback loop
- Govern model calls, spend, sensitive data, and human approval
- Offline and online evaluation
- Human feedback and annotation queues
- LLM-as-judge, heuristic, pairwise, and custom evaluators
- CI/CD evaluation integration with pytest, Vitest, and GitHub workflows
- Framework-agnostic and SDK/API-based
Categories
- AI tools
- LLM evaluation tools
- LLM observability