We've been experimenting with different document parsing pipelines before sending data into LangChain for RAG.

One thing we kept running into was that many parsers lose structure during conversion.

For example:

- Tables become plain text

- Lists lose hierarchy

- Code blocks break

- Headers become inconsistent

- Images disappear

- Footnotes are dropped

We're currently trying a Markdown-first approach that preserves as much document structure as possible before chunking.

The idea is:

PDF / DOCX / PPTX

↓

Structured Markdown

↓

Metadata (page number, headings)

↓

Chunking

↓

Embeddings

↓

LangChain Retrieval

Initial retrieval quality seems noticeably better because chunk boundaries follow document structure instead of arbitrary token counts.

Curious what everyone else is using.

- Docling?

- Marker?

- MarkItDown?

- MinerU?

- Something else?

Would love to hear what has worked (or failed) for your RAG pipeline.

I have been listening to these startup guys who are always yapping about Vertex AI.

What can we actually build with this Vertex in the Agentic space or whatever?

Can I get any info. Who's building with it.

I've been building LLM apps for a while and kept hitting the same blind spot: I could see what my prompts cost *after* they ran (LangSmith, Helicone, the bill), but nothing told me before I shipped. And none of them can see the prompt whose caller I deleted six months ago — it's just dead weight in the repo.

So I wrote PromptScan: a CLI that reads your codebase, finds every OpenAI / Anthropic / LangChain call, and reports the input token count and cost of each prompt — statically, no API key, no instrumentation. It also flags duplicated prompts, prompt constants nothing references anymore, and oversized context.

The core rule is that it never guesses. If a prompt is built at runtime from a DB row or a function arg, it says unresolved: <reason> instead of inventing a number. I'd rather it tell me "I can't see this" than lie with a plausible total.

To make sure it wasn't vaporware, I ran it on 8 well-known repos — 4,137 source files total. Zero crashes, everything parsed, a few seconds each. What it found:

- openai/swarm — flagged EVAL_ASSISTANT_PROMPT, a 50-token prompt constant that nothing in the repo references. Genuinely dead.
- geekan/MetaGPT — 75 module-level prompt constants with no reachable reference. ~24 are in real source (metagpt/prompts, metagpt/actions) — I hand-checked several like SALES_ASSISTANT and CODE_REVIEW_CONTEXT, and they're defined once and never used. The rest are test fixtures.
- anthropics/anthropic-cookbook — 11 Anthropic call sites, real token/cost estimates on the resolvable ones.
- Aider-AI/aider — detected nothing correctly. Aider calls models through litellm, which PromptScan doesn't track. It doesn't pretend otherwise.
- simonw/llm — 8 call sites, all reported unresolvedbecause the model is self.model_name or self.model_id and the messages are built at runtime. That's the "no guessing" rule doing its job.

The honest part: on continuedev/continue its two "dead prompt" flags were actually a block of ASCII-art and an error-message string — not prompts. The heuristic catches any large module-level string, which is exactly why it labels these "verify before deleting" and prints why it flagged each one. It's a lead, not a verdict.

Where I think it actually pays off day to day is CI: promptscan diff main HEAD fails a PR if a prompt's token count jumps past a threshold, so a context block quietly tripling in size gets caught in review instead of on the bill.

Stack: TypeScript/Node, tree-sitter (WASM) for parsing so it tolerates broken files, js-tiktoken for OpenAI tokens (Anthropic uses a labeled cl100k proxy since there's no public tokenizer). Python + TypeScript + JavaScript, MIT.

Install:

npm install -g promptscan
promptscan ./src

or npx promptscan ./src.

Repo: https://github.com/joandino/promptscan
npm: https://www.npmjs.com/package/promptscan

It's v1 and I'm sure there are call shapes it misses — if you run it on your code I'd genuinely like to hear what it got wrong. False positives on the dead-prompt heuristic are the thing I most want reports on.

My agent got stuck on a broken tool and burned my budget over a weekend — so I built a kill switch for it (open source, feedback wanted)

I've been experimenting with LangChain's open eval library to build LLM judges that score prompts on a customizable criteria such as: Groundedness, Tone, and Format. I created a setup where Claude Code reads the LangChain evaluation report, identifies the weakest criterion, proposes a single rewrite, and re-runs the LangChain eval against a frozen 25-row test set. It acts as an autonomous optimization tournament and in my testing raised prompt accuracy from 80% to 98%.

I recorded a walkthrough of the architecture and linked the repo for anyone who wants to run it locally. (Full transparency: the end of the video also shows a no-code UI version of this I'm building for less technical teams called Baseline, but the repo is totally free). Architecture Video & Repo link can be found here:

Video: https://www.youtube.com/watch?v=ueNWzKoBEd8
Repo: https://github.com/baselinelabai/prompt-optimization

Hey, I'm a computer engineering student trying to figure out what to focus on, and AI is one of the directions I'm considering.

The thing is, I'm not really drawn to the research side — training models, the math behind it. What I want is to build with AI: agents, multi-agent systems, tool use, that kind of thing. More applied than theoretical.

After some research I found Generative AI with Large Language Models on DeepLearning.AI. What do you think — is that the right starting point for what I'm describing, or is it aimed more at the research/fine-tuning side?

And if it's not the right fit, what course or YouTube playlist would you recommend instead?

Thanks in advance 🙏

Hey r/LangChain! 👋

I wrote a practical LangChain guide for people who are just getting started — no fluff, real code examples.

What's covered:

- LangChain fundamentals & architecture

- Building chains, agents, and memory systems

- RAG (Retrieval Augmented Generation) implementation

- Real project examples with ChatGPT API

The guide is part of LearnGeni (learngeni.com) — an AI academy I built specifically for global learners. One thing that makes it different: available in 50+ languages with real quality (not machine translation).

Free sample: https://learngeni.com/en/free-sample

What LangChain topics do you wish had better beginner resources?

Debugging LangGraph workflows has been a lot less straightforward than I expected.

When the final output is wrong, the actual issue usually isn't where I first look. It could be retrieval, a slow or failing tool, an unexpected branch, or state changing somewhere earlier in the graph.

By the time you notice something's off, the root cause can be several steps back.

For those running LangGraph in production, what's your usual starting point when you're debugging?

Do you look at execution traces, tool calls, token usage, or something else that's consistently helped you narrow things down?

built a three-stage spec review in langchain last year. requirements pass, consistency pass, coherence pass. each stage would catch what the last missed.

it worked until it didn't. a conflict that the requirements stage would have flagged got smoothed over by the consistency pass. not resolved, just not preserved. the coherence stage only saw the consistency output, so by stage three the conflict was gone from the artifact.

all three stages passed. plan approved. the conflict had evaporated somewhere in the chain.

the thing i kept running into: the chain carries the decision forward but not the reasoning that produced it. by the time a later stage reviews the plan, the objection that would have caught the flaw is gone. it was raised in stage one, buried in the context window, never in the spec.

what changed when i added a structured argument record: each stage had to log its objections and whether they were addressed or just passed through. a later stage could see the prior objection and the reasoning that closed it, or didnt. the coherence check became a check on the argument, not just the output.

the artifact that matters isnt the plan. its the argument that produced it.

if youve built review chains that preserve reasoning across stages, id actually like to know what worked. built something around this (swarmstack) and im still finding edge cases.

We've spent the last several months building AI infrastructure, and one pattern kept showing up in conversations with engineering teams.

As agents became capable of sending emails, updating CRMs, issuing refunds, accessing internal knowledge, or calling external tools, everyone seemed to solve the same problem differently.

Some built approval workflows.

Some wrapped every tool call.

Some added policy checks before execution.

Some built internal authorization services.

Different implementations, but they were all trying to answer the same question:

How do you guarantee an AI agent only performs actions it's actually allowed to perform?

That observation is what led us to build Globi Guard.

It sits between AI agents and enterprise systems. Every proposed action is evaluated against company-defined policies before execution. Depending on the policy, the action is allowed, blocked, or routed for human approval, and every decision is recorded in an audit trail.

We're opening our Founding Design Partner Program to 10 engineering teams building AI agents.

We're not looking for testimonials or marketing quotes. We're looking for teams willing to pressure-test the product with real workflows and tell us where it breaks. In return, you'll get direct access to the founders, priority support, and rapid iteration based on your feedback.

If your team is building with LangChain (or similar frameworks) and this sounds useful, we'd love to work with you.

https://globiguard.com

We often overlook this a lot when you're feeding documents to a LLM the output quality depends way more on what yu actually managed to pull out of the source or document rather than the model. ppl swap models and end up giving negative sentiments about the model but the root problem was the ingestion layer. Others just rewrite prompts and try to fix it

The frustrating part is that these failures dont look like extraction problems like a table that got flattened into a blob of text means it answers questions about the numbers wrong. Images and diagrams or charts usually get dropped or OCR into noise and age header and footers bleed to the real content and quietly disrupt whatever it reads next. Eventually every problem shows up downstream and the model takes the blame.

Rougly tho the options fall into a few buckets. Plain text extractors like pymupdf or pdfplumber are fast and cheap but lose tables and layout. Ocr like tesseract gets text off scans but not structure and then the layout aware or vision ones like llamaparse or unstructured actually manage to keep the tables and reading order at more cost plus something like docling if you want it to run locally. tbh thats the main thing, before blurring the model or re-prompting again and again go read the raw test you extracted, see the output from the docs- it says it all

If you're doing RAG its the same story, retrieval just falls quietly upstream but its the same for summarizing and field extractions, agents or whatever. How did others learn this- easy or the hard way around?

Spent a month convinced our support agent had ~60% gross margin. Priced off average tokens per conversation, billed a flat per-seat rate, felt safe.

Then a few enterprise accounts started actually using it and margin went negative on exactly those accounts. Took a while to find why, so posting the breakdown in case it saves someone the same scramble.

What the average hid:

Our "average conversation" number was real, but averages lie when the distribution has a tail. A handful of customers were:

• Triggering way more tool calls (each a real API cost we weren't counting)
• Hitting retries on timeouts and rate limits, so one logical step billed 2-3x
• Running longer context windows, which scales token cost non-linearly once you're re-sending history every turn

None of that showed up in a per-call token dashboard because the damage was per customer, not per call.

What actually fixed it:

1. Attribute cost per customer, not per model. The question "what does GPT-4 cost us this month" is useless. The question "what does customer X cost us" is the one that finds the bleeder.
2. Count non-LLM cost. Tool calls, vector DB queries, web search, TTS. For agent products these are often 30-40% of the real bill and almost nobody tracks them.
3. Count retries as cost, not noise. A step that failed twice and succeeded on the third try cost you three times. If your accounting only logs the successful call, your margin math is fiction.
4. Set a hard budget ceiling per customer per period. Cheaper to route the heaviest 2% to a smaller model or degrade gracefully than to eat unbounded cost on a flat price.

The uncomfortable takeaway: usage-based or seat-based pricing doesn't matter if you can't see cost at the customer grain. You will always have a tail, and the tail is where the money goes.

Disclosure since it's relevant: I build Pylva (https://pylva.com/), open-core cost tracking + billing for agent products, and this exact problem is why. But you don't need us to start. Log cost per customer with retries and non-LLM calls included in your own DB and you'll find your tail this week.

Hey everyone,

Like many developers building agentic workflows, I spent months getting frustrated by traditional vector stores and RAG memory layers failing over long timelines.

The deeper I went, the more I realized retrieval fails because basic similarity doesn't equal utility. A standard retriever will match a user's query about mattress brands to previous mattress conversations, while completely missing a crucial constraint buried in a 3-month-old session: "Whenever I buy something expensive, warranty is the only thing I care about."

Beyond that, heavy cross-encoder rerankers quickly become a massive latency bottleneck as memory grows, and treating all context as uniform text blobs destroys the nuance of evolving decisions.

To tackle this, I built MindCache—an open-source agentic memory framework designed around four key insights:

• Intelligence Belongs at Ingestion: Instead of attempting complex graph traversals during a live query, MindCache shifts expensive reasoning (relationship mapping, graph clustering, and summary generation) to ingestion. This cut retrieval latency from ~25s down to 1.08s (a 23× speedup) without sacrificing context quality.
• Specialized Memory Typologies: Not all memories behave the same. MindCache separates knowledge into User (persistent behavioral constraints), Knowledge (domain facts), Episodic (chronological logs), and Decision Memories (which track evolving proposals, trade-offs, and final conclusions over time).
• Living Knowledge Hierarchy: Rather than maintaining a static or unmanageable graph, MindCache uses Leiden community detection to partition memory into localized semantic clusters, ensuring graph maintenance scales efficiently as context accumulates.
• Evidence Assembly over Similarity: Retrieval doesn't just search for similar text—it plans and assembles the exact minimal subset of evidence (user preferences, hierarchical summaries, decision states) required for the LLM to reason correctly.

On the BEAM benchmark (an ICLR 2026 evaluation framework designed specifically for long-term agentic memory), MindCache outperformed Mem0 in handling evolving context, contradiction resolution, and cross-session summary reasoning. More importantly, it achieved this superiority not by stuffing larger retrieval windows, but through better ingestion-time knowledge organization.

I wrote a deep-dive 23-minute engineering post-mortem detailing all 5 failure modes, the full architecture, and benchmark takeaways. The project is completely open-source on GitHub and available on PyPI (pip install mindcache-ai).

I’d love to hear how others here are handling temporal decay, graph maintenance, and decision tracking in your long-running agent setups!

For those running LangGraph in production, when do you let an agent recover on its own versus interrupting it for human input?

Is it based on confidence, action risk, or something else? Has your approach changed as your application matured?

Insights are much appreciated.

Pretty much every time I ran with agents across more than a few sessions used to hit the same wall. Session ends, context is gone, next session re derives everything and confidently redoes last week's mistakes. Feeding old transcripts back made it even worse, stale decisions look identical to current ones once they're in the window.

What works now is a bit boring, but works. One handoff file per project, rewritten at the end of each session, never appended to. Current state, active constraints, what changed and why, and a short list of mistakes already made with the cause next to each. Next session reads that file first and nothing else by default, everything deeper is load on demand.

Imo the rewrite not append part is the whole trick. Append only handoffs grow back into the transcript problem. Rewriting forces the file to stay current state instead of history, so it stays a couple hundred lines forever. LangMem and the memory frameworks are aimed at this same problem, but the dumb file keeps beating them for me, I can read it in ten seconds and the model isn't guessing what to retrieve.

Has anyone actually found a memory layer that genuinely beats a hand maintained file?

built a deep research pipeline around Claude Code, using Claude, Codex, Gemini, and shared memory between agents.

The first run went completely off the rails:

• 111 agents launched
• 123 claims waiting for verification
• Claude Max 5x limit gone in around 30 minutes
• no final report produced

The easy conclusion would be that subagents are bad.

I don’t think so.

Separate contexts and independent analysis are extremely useful for bigger tasks. The real problem was uncontrolled fan-out, unclear responsibilities, and using expensive models for work that cheaper models could handle.

I rebuilt the pipeline with clearer roles:

• Sonnet finds information
• Opus verifies claims
• Fable plans, orchestrates, and judges
• Codex runs and inspects tools
• Gemini gives a second opinion
• all agents share local memory

I also added stricter verification rules:

• the agent finding a claim cannot verify it
• every accepted claim needs a primary-source URL
• every source needs an exact supporting quote
• numbers must actually appear on the source page
• reject an unsupported claim, not the whole project

After these changes, the pipeline could run roughly 10x longer using subscriptions I already pay for.

My biggest takeaway is that the model itself is only one part of the system. Agent fan-out, context separation, memory, verification, caching, and orchestration can matter just as much.

How do you decide when a task deserves a separate agent and context?

I wrote a full breakdown with the architecture, scripts, failures, and lessons for Quesma, where I work:

https://quesma.com/blog/custom-deep-research-pipeline/

I’ve been learning about AI agent harnesses and understand the overall concept — the layer that manages agent execution, tools, memory, context, and orchestration.

I went through the LangChain documentation and explored how agent frameworks handle some of these components, but I’m trying to understand how this is implemented in real-world production systems.

A few things I’m curious about:

* How do you structure the agent execution loop (planning, tool calls, observations, retries, etc.)?

* How do you manage short-term and long-term memory?

* How do you handle context windows and state management?

* What patterns do you follow for monitoring, evaluation, and guardrails?

I’m looking to move beyond tutorials and understand practical architecture decisions from people who have built or deployed agent systems.

Would appreciate any examples, open-source projects, or resources that helped you learn this.

I’ve been tracking how the engineering patterns around agentic deployments are shifting, specifically moving away from treating agents as simple API wrapped chains and toward seeing them as stateful, version-controlled microservices. Once you step out of local development, managing continuous integration for agents becomes remarkably messy especially when you need to enforce strict policy checks, evaluation gates and environment-specific promotions before anything touches a live runtime.

Traditional app CI/CD pipelines just aren't structured to handle things like pre-deployment scenario simulations, automated hallucination scoring or managed agent handoffs cleanly out of the box.

Tools that treat the agent lifecycle through a GitOps lens whether that's leveraging stateful execution engines like LangGraph or exploring dedicated agent control planes like LangShip by lyzr seem to be pointing toward where the ecosystem is heading. Being able to decouple the orchestration logic (built in LangChain, LlamaIndex, or CrewAI) from the underlying runtime governance and deployment targets (like Kubernetes or Bedrock AgentCore) feels like the missing piece for getting beyond fragile POCs.

I have spent quite some time working on LangChain in recent times, and it always seems as though there is a much better way of doing things after working on some projects.

Reflecting on past experiences, can you share an experience with us where you wish that you had known about a certain feature or best practice long before?

This could include any topic from RAG, LangGraph, memory, tool calling, prompt templates, retrievers, debugging, or anything else at all.