Made withLlamaIndexTutorial

A quick walk-through of LlamaParse: simplified document parsing for generative AI applications

L

LlamaIndex

Creator on the LlamaIndex channel

Watch on YouTube ↗

14K viewsJul 23, 202415:23 runtime

Discover the power of LlamaParse for simplified document parsing in your generative AI applications. This comprehensive guide walks through key features, from the web UI to programmatic usage in Python and TypeScript, and direct API calls. Learn to handle diverse file types like…

Chapters

#llamaindex#tutorial#aillm

Getting Started with LlamaParse

LlamaParse offers several ways to parse your documents for generative AI applications.

Web UI: The simplest way to start is through the LlamaCloud UI at cloud.llamaindex.ai. It provides a drag-and-drop interface with options for language, page selection, caching, and more.
Python: To use it programmatically, install llama-parse and llama-index-core. Set your LLAMA_CLOUD_API_KEY and instantiate the LlamaParse object. It integrates seamlessly with SimpleDirectoryReader.
TypeScript: Install llama-index and set your API key. LlamaParse is built into llama-index-ts and can be used via the LlamaParseReader.
Direct API: For other environments, you can call the REST API directly using cURL or any HTTP client to upload a file, check parsing status, and retrieve the results.

Supported File Types & Output Formats

LlamaParse is not limited to just PDFs. It supports a wide variety of document types:

PDFs
Microsoft Word (.docx)
Microsoft PowerPoint (.pptx)
Images
Spreadsheets (Excel and others)

Note
LlamaParse can return documents in three main formats: plain text, Markdown, or JSON. The choice of format depends on the complexity of your document and your application's needs.

Advanced Parsing with JSON Mode

For documents with rich structure, JSON mode provides the most metadata.

Multimodality: The JSON output includes objects for elements like images. You can extract these image references and create multimodal indexes, allowing your RAG system to query both text and the content of images (e.g., by generating text descriptions for them).
Structured Data Extraction: The LlamaParseJsonNodeParser (available in Python) can process this JSON output. It creates structured nodes, such as table objects, which can be used with a RecursiveRetriever in LlamaIndex. This enables your application to query the contents of a table directly, rather than just treating it as plain text.

Important
Using the LlamaParseJsonNodeParser allows for more accurate and context-aware retrieval from complex documents, as the system understands the document's underlying structure (e.g., what is a table vs. what is a paragraph).

Leveraging the Markdown Node Parser

When parsing to Markdown, you can use the MarkdownElementNodeParser. This parser takes the Markdown output from LlamaParse and breaks it down into structured nodes based on Markdown elements (headings, lists, tables). This provides a richer representation of the document than raw text, improving the quality of your RAG system.

Specialized Parsing Features

LlamaParse includes features to handle non-standard and complex documents.

Multi-language Support: By specifying a language (e.g., 'fr' for French), LlamaParse improves its OCR and text extraction for non-English documents, correctly handling special characters and accents.
Parsing Instructions: This powerful feature allows you to give LlamaParse hints about the document's content and structure. For example, you can tell it:
- A document is a manga that reads right-to-left to reconstruct dialogue correctly.
- To output mathematical formulas as LaTeX for perfect rendering and semantic understanding.

Tip
Use parsing instructions whenever you are dealing with unconventional layouts or specialized content to dramatically improve parsing accuracy.

Introduction to the LlamaParse UI

Hi everybody, I'm Lora, I'm VP of developer relations at LlamaIndex and I'm here to talk to you today about LlamaParse. We're going to be doing a quick and dirty walk-through of some of LlamaParse's enormously great array of features. So let's first talk about how you get into LlamaParse, which is you just go to cloud.llamaindex.ai and you log in.

There's a parse section and there's a convenient little UI here for you to try out basic parsing. You can see that there's a huge list of options. There's language, there's parsing instructions, there's page separators, prefixes, suffixes. You can choose specific pages of your document to parse, you can set a bounding box, you can skip diagonal text, you can invalidate your cache, or you can avoid caching entirely. There's a whole bunch of stuff that you can do, and we're going to be talking about those later on in this video. In the meantime, let's just try dragging and dropping a file, in this case a PDF guide to San Francisco, and we'll hit parse.

It takes a little while to parse, but then you get in this version a Markdown set of text, giving you Markdown tables, headings, all that kind of stuff showing the contents of the PDF. If you don't want Markdown, just turn off the Markdown mode and you will instead get text which includes the original columns and all the rest.

Programmatic Usage in Python and TypeScript

So let's talk about how to use LlamaParse in other ways. Programmatically is how you'll be using LlamaParse most of the time. This exploration by the UI is mainly for getting started.

Using it in Python is pretty easy. You set a Llama Cloud API key, you install LlamaParse along with LlamaIndex core, and then you bring in LlamaParse and in this case SimpleDirectoryReader. And you create the parser, in this case we've set result_type to Markdown. You tell SimpleDirectoryReader to use an extractor for PDFs, you give it the input files, and then you tell it load_data. Then you print your documents and you'll get your parsed file.

You can also use TypeScript. TypeScript is much the same. You set the same API key, you install TypeScript, you install LlamaIndex. In this case, LlamaParse is built into LlamaIndex.TS. And you import LlamaParseReader, then you create a function that creates the reader. You call reader with reader.load_data with your path and you get your parsed document.

Using the LlamaParse API and Documentation

You can also use it via the API. If you don't want to use Python or JavaScript, you can call it directly from the API. This is showing some cURL calls. This is how you'd upload a file to start parsing in the first place. So you pass in api.cloud.llamaindex.ai and the API endpoint for parsing. You pass in your API key as your authorization bearer and you pass in a path to your file. You can use this endpoint to check the status of your parsing job, because parsing can sometimes take a while if it's a very large job. And then you can get the results just like you did in the UI and in the two, in JavaScript and Python like we just said.

We have full API documentation. Whoops, I should fix that link. You have full API documentation down here showing all of our endpoints. So every single one of them is documented with full examples in cURL, Python, all of that kind of stuff. Your full API for all of LlamaCloud is documented here.

Supported File Types & Excel Parsing

So now let's look at some examples in this repo at LlamaParse examples. We have a huge set of examples of how to use LlamaParse. Let's just go through a couple of them.

Before we do that, I wanted to mention how many different types of files we support. I've only been using PDFs because that's our basic type, but we support all sorts of document types like Word docs, PowerPoints, all sorts of obscure formats that you've never heard of. It can parse images and it can parse spreadsheets, Excel spreadsheets, and a huge array of other types of spreadsheets. So really, LlamaParse can do a whole bunch of parsing of all sorts of document types, not just PDFs.

Here's an example of it parsing an Excel file. It's exactly the same as parsing any other file. We've created our parser with an API key, we've told it to give us markdown, we've said load_data, this time on an Excel file, and you can see it's given us a beautiful text table of the data in the spreadsheet.

The rest of this is about how to build RAG with those documents, which we're not going to be talking about today.

Advanced JSON Mode for Multimodal Applications

Now let's talk about JSON mode. One of the things that you can do with the API is get JSON back, a JSON document structure representing your parsed document, not just markdown or text. In this case, we are setting up JSON mode using LlamaParse and we're using the get_json_result method to get the raw JSON.

What we're doing here is we're going to use the fact that the JSON mode contains more metadata than text or Markdown would give you. One of the things that the JSON mode gives you is it gives you image objects in the JSON. So we have a function here which looks at our JSON extracted and finds all of the images that are referred to in the JSON mode and turns them into image documents, which is LlamaIndex's way of dealing with multimodal images.

You can then create a vector store index that contains both your text nodes and your image text nodes. And you can do query engines that look up across your images. In this case, what we did was we attached metadata to each image using an LLM to describe what the image was. So then we got text nodes that said what the actual what the images were, and we use that in the index to look stuff up.

Using the JSON Node Parser for Structured Data

The next cool thing that you can do with JSON mode is you can do, you can use the LlamaParseJsonNodeParser, which is a built-in from LlamaParse in the Python version. What this does is it parses complex documents with things like tables and images and stuff like that from the JSON mode into a richer format.

Specifically, what it does is, you get JSON results like you did before. You can see this is what the JSON result looks like. And then you pass that output to the LlamaParseJsonNodeParser. So we get the nodes from the documents and we pass it to the node parser. What this does is it gives us, allows us to give our set of nodes and recursive nodes. So we don't just have the text of the document here, we have table objects that we can then give to our recursive retriever. So a recursive retriever in LlamaIndex allows you to do queries across your entire document that if it returns like a table object, instead of just saying here's the text of the table, it knows that it is a table and it can look into the table and give you specific answers from that table, which is what it is doing here. It's asking what is Uber's short-term Insurance reserves reported in 2022? That answer is living in a table, and our retrieval is able to pull it straight out of the table because of the rich metadata that it has about the tables that it got from JSON mode.

Leveraging the Markdown Element Node Parser

Another thing that you can do when you have complex tabular data is you can use a recursive retriever using the MarkdownElementNodeParser, which is what we're doing here. So in this case, what you do is you use LlamaParse with result_type equal to markdown to read some documents, and then you feed those documents to the MarkdownElementNodeParser. So you can see us instantiating the node parser here, and then we pass each of these things to the node parser.

This allows us to create a query engine for each of these documents that is able to provide rich, again rich information about the contents of the document because we have so much more information from LlamaParse than we would have if we just parsed raw text.

Multi-Language Support for Global Documents

Another feature that you might have seen when we were looking in the UI at the options is multi-language support. OCR and parsing over non-English languages is often quite fraught because of different character sets and things like that, and that is what this feature is intended to take care of.

So in this case, you can say LlamaParse with a result type text and the language is French, parsing a French document. You get, you know, accents and things like that that you would not otherwise get because LlamaParse knows what language it's working in.

Advanced Control with Parsing Instructions

And the final feature that I want to talk about today is parsing instructions. Parsing instructions are an incredibly rich feature. Basically, parsing instructions allow you to tell LlamaParse what kind of document it's looking at before it parses it.

So in this case, we've downloaded a manga, a comic book, and we're parsing it with, in this example we're parsing it without parsing instructions, and you get really weird results because, a magazine, you know, it's got these big boxes that have text in balloons for it and it reads right to left instead of left to right.

So instead, you create some parsing instructions: 'The provided document is a manga comic book. Most pages do not have a title. It does not contain tables. Try to reconstruct the dialogue.' And you pass those parsing instructions as a parameter to LlamaParse when you're instantiating it. And suddenly it knows what it's looking at. It tells you, you know, it gives you the dialogue in the right order.

And it can also do other stuff like it can parse math. This manga happens to be about math, so we don't have to go and find another PDF about math. But as you can see here, it has a bunch of complex equations with mathematical notation, and you can give it the instruction that you should output any math equations in LaTeX markdown, because our parser knows what LaTeX is. And here you get, you get your equations turned into into LaTeX instructions here. This is without instructions, you get, it does its best but it's, you know, it's obviously converting math into text as best that it can. But this is full LaTeX that renders out perfectly. This is a rendered version of our LaTeX instructions parsed out of the manga. As you can see, it looks almost exactly the same as the original manga, it's just not in Comic Sans anymore.

So that's our top-level list of examples of LlamaParse. This has been quick and dirty, but I hope you can see that LlamaParse has a huge body of features and a whole bunch of stuff that you can do. And thanks for your time and attention.