LangSmith

LangSmith is a software-as-a-service platform developed by LangChain, Inc. for tracing, debugging, evaluating, and monitoring applications built with large language models (LLMs) and AI agents. Although it was originally introduced as the production companion to the LangChain framework in 2023, it has since been generalised into a framework-agnostic observability layer that ingests traces from the OpenAI SDK, the Anthropic SDK, LlamaIndex, custom application code, and OpenTelemetry.

What problem it solves

LLM applications are notoriously hard to debug. A single user-facing response may be the result of dozens of prompt templates, retrieval calls, tool invocations, and agent decisions, each of which contributes latency, cost, and potential failure modes. Conventional application performance monitoring tools record HTTP requests and CPU metrics but cannot expose the prompt that was actually sent, the retrieved context, or the reasoning trace of an agent. LangSmith fills this gap by recording the inputs, outputs, intermediate steps, token usage, and metadata of every LLM and tool call, and by attaching them to a hierarchical trace that mirrors the execution graph of the application.

Core capabilities

| Capability | Description | |---|---| | Tracing | Hierarchical capture of LLM, retrieval, and tool calls with inputs, outputs, latency, and cost | | Monitoring | Dashboards for latency p50/p95, error rate, token spend, and custom metrics | | Evaluation | Run datasets through chains and grade with LLM-as-judge, exact match, or human review | | Prompt management | Versioned prompt registry with commit, tagging, and rollback | | Annotation queues | Route traces to subject-matter experts for labelling | | Dataset curation | Build evaluation sets from production traces of interest | | Online evals | Continuously score live traffic on quality, safety, and policy compliance | | Alerting | Notify on regressions in cost, latency, or quality |

Instrumentation can be as light as wrapping a function with the @traceable decorator in Python or traceable in TypeScript, or as deep as exporting OpenTelemetry spans from any service. Traces include cost estimates for hundreds of model SKUs, which helps engineering teams understand spend across providers.

Evaluation workflow

A typical LangSmith evaluation cycle begins with a dataset of inputs and expected behaviours, often bootstrapped from production traces. Engineers define evaluators, which may be heuristic (exact match, regex, BLEU), embedding similarity, or LLM-as-judge prompts that score correctness, helpfulness, and faithfulness. Experiments are run by replaying the dataset through one or more versions of the chain or agent; results are compared in a side-by-side view that shows per-example scores, latency, and cost. This pattern lets teams detect regressions before they ship and supports continuous improvement of prompts and retrievers.

Deployment options

LangSmith is available as a fully managed cloud service, a Bring-Your-Own-Cloud (BYOC) deployment that keeps trace data inside the customer's account, and a self-hosted option for organisations with data-residency or regulatory constraints. The self-hosted edition is commonly chosen by financial services and government workloads in regulated markets.

Competitive landscape

LangSmith competes with Langfuse, Helicone, Arize AI, Phoenix (Arize), Weights and Biases Weave, Galileo, Honeyhive, and OpenTelemetry-based open-source stacks. Differentiation comes from depth of integration with the LangChain and LangGraph ecosystems, the maturity of evaluation tooling, and prompt management features.

Malaysian organisations building generative AI applications increasingly need production-grade observability to satisfy internal risk, audit, and regulatory expectations. Bank Negara Malaysia (BNM), the Securities Commission (SC), and the forthcoming Malaysian AI Governance Framework coordinated by MOSTI and the National AI Office expect operators of AI systems to maintain audit trails of model inputs and outputs, traceable explanations for decisions, and continuous monitoring for hallucination, bias, and policy violations. Platforms in the LangSmith category map directly onto these expectations.

Local fintechs, digital banks (GXBank, AEON Bank, Boost Bank, Ryt Bank, KAF Digital Bank), and e-commerce players use LLMs for customer service triage, KYC document understanding, and internal knowledge search. These workloads typically pipe traces into observability platforms to monitor accuracy and cost. The PDPA, enforced by the Personal Data Protection Department (JPDP), restricts cross-border transfer of personal data, so the BYOC and self-hosted deployment options are particularly relevant for Malaysian operators handling MyKad numbers, financial records, and health data.

System integrators and AI consultancies based in Cyberjaya, Bangsar South, and Penang (including Fusionex, Silverlake Axis, Securemetric, Innov8tif, Naluri, and the Hermes Agent / Teragrid Agent partner ecosystem under AITG SDN BHD) increasingly bundle LLM observability into their delivery stack. MDEC and HRD Corp support relevant upskilling through claimable training programmes covering LLM evaluation, prompt engineering, and AI quality assurance.

See Also

References

1. LangChain, Inc. LangSmith Documentation.
2. LangChain, Inc. (2024). LangSmith General Availability Announcement.
3. OpenTelemetry Project. Specification for Generative AI Conventions.
4. Bank Negara Malaysia. (2024). Discussion Paper on the Use of AI by Financial Institutions.
5. MOSTI. Malaysian National AI Roadmap and Governance Framework Consultation Materials.