Name: End-to-End Neo4j Developer Workspace with MCP with Alex Gilmore & Jean-Marc Guerin, Neo4j
Uploaded: 2026-03-26T19:16:09.679Z
Duration: 2 h 3 min 4 s
Description: End-to-End Neo4j Developer Workspace with MCP with Alex Gilmore & Jean-Marc Guerin, Neo4j

Transcript for "End-to-End Neo4j Developer Workspace with MCP with Alex Gilmore & Jean-Marc Guerin, Neo4j": Alright. Good morning, good evening, good afternoon, and welcome to our second to last Notes AI Road to Notes AI workshop. I'm I'm very oh my know, it's a it's a sad sad moment and happy moment at the same time. So that we have this it's an amazing series of workshops running, and this is this is the second to last one, but it's a it's an it's an exciting one. I'm very much looking forward to into a Neo four j developer workspace with MCP. And I'm happy to have Alex Gilmore, who is an AI solutions architect at Neo four j, and Jean Marc Gurin, who is also an AI solutions architect here at Neo four j together. They'll they'll guide us through this workshop. Super super happy to to have them both here. They are the experts on the on the scene. Obviously, everybody's talking about MCP. It's it's a hot topic. It's very interesting to many people what to do with it, how to how to get going. So I'm I'm very excited to hear what what their take is on on the whole thing. And if you have any questions, anything is unclear, anything you would like to dive in a little bit deeper, please use the q and a section of the chat. So you should see three, you know, buttons sort of on top of the chat window, and one says q and a. Please send your questions in there. If you use the chat, we might miss your question. So it's better if you use the q and a button so we can have a little bit of a of a better overview. But, obviously, you can also chat away in in chat. I see people from France. I see people from Mexico, from India, from The United States, and more. So Argentina here, Spain. I think I said Spain. So welcome, everybody, and happy to have you join today. And one more thing, this session is obviously recorded. So if you're wondering, oh, what's going on? I need to jump out, but I wanted to watch it later on or you missed something because it was maybe a little bit quick. We'll send you an email afterwards with all the information that was covered today and, obviously, with a link to the recording so you can watch that at any time after after we're done today. But, yeah, I I think I think that's that's covered my bit. I'm super happy to to get this show on the road. Jean Marc, you need to click on go on stage, which is a button that you see bottom right, I think, in your screen. So you'll be appearing up here with us? here. Hello. Hello, Fred. Not a problem. And, yeah, I'll hand it over to you, Jean Marc. Alex, take it away. I'll be here. Jennifer is also here as a as a moderator and help helps out with questions. So we'll we'll be here for for the time of the workshop. And I, yeah, wish you all a good time and and see you see you on the other side. Alright. Thank you, Alex. And, well, welcome. Good morning, good afternoon, good night. So maybe, Alex, yeah, you can share the slides. So today, we have two hours together, we're going to share with you what we call the MCP workspace, a Neo four g MCP workspace. So this is a repo we just published on GitHub. Hopefully, it will be useful for you. And maybe next slides. Yeah. So for today, we have what we think will be a one hour presentation of so quick introduction about why do we want to build a graph and what we can do with graph, especially in collaboration with GenAI. And then we will go deeper into an overview of the workspace that we built. And now it's available in GitHub again. And then so feel free to ask questions and make them as interactive as possible. Then we prepared two demos. So let's see if we have time to show both demos or only one depending on the questions. But anyway, we already recorded the demo as a backup plan, and we already shared and we will show you we will drive repo with both the slide deck and the video demo and the data that we will use in the demo. And, of course, with the GitHub repo so that you can watch it later and and reproduce the the full exercise by yourself. And then, yeah, of course, we'll have open workshop and questions. And, yeah, and feel free again to to ask question directly here. And next. So what's the idea of today? The first step is to understand graph and graph hag and the methodology that we would recommend to build graph. So why a graph schema matters and how we recommend building one especially when you have structured and unstructured data or unstructured data only. And this is exactly the purpose of the workspace, is to help you and mostly automate the graph construction from the beginning, which is understanding the data to the end, testing the graph with different questions or queries that we want to generate. So we will also show how I mean, the workspace, how it works. So the workspace, what it is, it's a GitHub repo that is available for ClotCode or Cursor. It should work with other type of agentic coders as well. But well, what we're going to demonstrate is first in Cursor and then in Clutecode, how it can work. And this is a graph development assistant. So basically, yeah, again, it's just automation here and some kind of interactivity that you will be able to use to to constrain the full workspace to build whatever you want to build with it. And so we'll also do on this on on exercises, mostly demos, but we will share with you the data and we'll show our our screen to show you how we use it and how it can be used either on demo datasets. And in the second phase, you may want to directly take your data, PDF or CSVs, and start building a graph by yourself. And so if you do so, please share your feedback or questions directly in the chat here and we will try to answer as much as possible. So we'll so you don't need any Neo four g experience with it to to use it. And because it will be used mostly in cursor or closed code, you don't need to write code by hand. Alright. Next. So the first step, why graph? So why Neo4j? Well, I'm sure if you're here, you already have at least a knowledge about what is Neo4j, what are graph and all those are useful. But yeah, this is more for an introductory slide. So the first thing is that we think that graph are a very good structure to connect all your type of data. Neo4j has been used for many years to convert structured data, so tables mostly, into a graph, building a relationship and nodes. And now with LLM and GNI in general, we find it very useful also to take your unstructured data and create a graph from your unstructured data. So we're we're going to see different ways to do that. And your unstructured data, it will be mostly text, but it can also be tables, images, or I work even with a video data, for example. Everything can be converted as a graph and linked to the user data that you may have in the database. So this is very good. And so also the very good point is that, yeah, so data so structure is also it's not just a representation. It's always using the database. And when you create new data into the database, it will be automatically linked if you follow, well, best practices that we're going to share. So the graph is really the integration layer here. And the second point is that well, this is why Neo four g was built, is that if you have a lot of well, now big companies have a lot of tables and the tables are very large. And when you want to query these tables, and you have complex questions that choose run across multiple tables, then it takes time and it takes CPU to do a lot of joints. I mean, it can be complex as well. While in Neo four g, when we create a graph, the relationships are first like first class citizen. And so they connect all your data together, and they are efficient, very efficient to craft traversal, of course. And so it's very easy to purchase this kind of knowledge layer that you built. So, yeah, we'll see different use cases. And the last point is that graph is a very flexible structure. So what we would recommend is to start small, use CSVs, your tables, use some parts of document, start to build the first graph with a simple schema and then grow. And graph are very good to evolve like that. Yep. And so the schema so the data will grow. The data that you're going to use to build the graph will grow. Your graph will grow in size and also your schema can grow. You can create new node types, new relationship types, and extract the new type of data directly and save that in the graph. The good thing is that once you have a graph, it can be used for several use cases. And what we see in the field is that, yeah, you can build a graph for a certain use case, and now we see more and more chatbot type of use case that wants to be able to query this graph into the different ways through LLM type of assistant. Next. So yeah. So why graph for Rag? So what we see also is that graph is a very good structure to build Rag type of agent retriever. So basically, when Rag started like two or three years ago, one of the standard was to use vector database. So you take your document, you cut them into pieces, so you create chunks. And for each chunks, you create an embedding, which is a vector representation of the content of your chunk that is searchable through similarity search, through vector search. So it's good to answer certain type of questions that are very precise. Basically, your answer need your question needs to be answer into one or several chunks, but it's difficult then to answer more complex questions like global questions or multi op questions. Something else is that VictorSearch, well, it's good to find similar pieces of text. So the idea is that you're going to embed your questions and find the chunks that are the most similar to the questions. But sometimes, it's not enough. For example, if I want to find in my database as as as a connection between my customer who purchased the same product, what I need is to do a similarity search of the product description and then traverse the graph to find information about the customers and ones that have been common. Okay? So for this type of questions, what you want is to combine semantic search, so vector search, plus keyword search sometimes. So what we call hybrid search, plus a cross traversal into one or several queries now with agents. And with that, you will be able to have more accurate results to your questions and also to answer more complex questions. Okay? Especially, so this is the second point here, multi op type of questions. So a graph can be built from your different chunks to connect the chunks that talks about the same subject or, yeah, combine them with the graph with entities and relationship. And your agent will be able to answer the question based not only on the chunks, but also on the graph itself. And this graph, because it links and because it was built from a lot of different checks, it's a very good way also to answer global type of questions. Okay? Things that are impossible to do with pure vector search. And the last point is that a graph, well, the way we build it, it will be explainable because we can source every note, for example, that was extracted to the source document or to the chunk. We can source the Cypher queries to to, well, exactly how it was searching the graph. The second point is that with that type of agentic HAG or graph HAG type of application, it's grounded because all the questions will be answered by information that is in the graph. And it's quite intuitive. This is what we're going to see. A graph is a good structure to visualize. It's not a black box like vectors, for example, that are impossible to understand with just your eyes only. So a graph, can explore it, discover it, and we have very nice visualization tools to do that. Next. And so I think here, yeah, I will let Alex introduce to well, so his subject here, which is the MCP workspace that we built together. Great. So let's see. Yeah. So just to kind of, like, intro the workspace here, what we based this off of was the typical, Neo four j graph, like, development cycle. So, really, what we typically see is we do some sort of data discovery upfront where we're looking at the source data, whether that's tables or CSVs or unstructured data. We're identifying the use cases that we want to address with our graph based on this data, and then we're going to create some sort of graph data model. This is our first version. We'll ingest our data based on this graph data model, write some Cypher queries that are based on the use cases that we identified upfront, and then see if those results align with what we expect. And, typically, what we're going to do next is provide corrections. We're gonna go back to the data modeling stage, make some tweaks, re ingest our data, maybe, change our Cypher queries a little bit. And this is an iterative process that we do. And so this is very common to see. This is kind of a inherent feature of graph development. And so we noticed that this fits really well with the sort of agent workflows that we were seeing, especially, like, with React style agents, which are which are very popular. And all of these tools and functionality, were able to be sort of abstracted with MCP. Right? So we can kinda put these all into different tool sets, put these into their respective MCP servers, and now we can kinda, like, compartmentalize these different steps of the process and expose this to the agent. The agent can then use this and kinda cover this whole workflow for us. And so kinda here, what I was getting at, you know, we have an iterative process at its core, that doesn't really have a defined number of times that we're gonna go through this, but we are typically gonna go through this process, at least a handful of times, to get our, you know, kind of final stage of our graph. There's a lot of tools that are available to us, especially in the AI space, depending on the data that we're working with, whether that's tables or unstructured data. And then what we want to you know, if we're gonna do knowledge graph generation, There's a lot of different tools out there for that specifically that we'll kinda, like, get into here. But, really, what we wanted to do is streamline this process. So we have an established workflow. We have an established tool set. We have skills, right, that we'll kinda talk about later that the agent understands how to do these processes that we defined and that we see out in the field every day. And so we wanted so we built this all into the workflow. And so the result of this is we have a set of MCP servers and agent skills that are compatible with Cloud Code and with the cursor IDE, and this allows for pretty fast, development of graphs. And that's whether you are a beginner. You know, this is like Jean Marc said, this is kind of a no code solution, at its core, and you don't need deep knowledge of Neo four j, to get something successful at the end of this. But also for advanced users, there is a lot of custom customization that you can do. You can kinda look under the hood. You can tweak this, specifically for, like, deep use cases, and you can kind of, implement your own expertise in this workspace as well, via, you know, like, own skills or your own MCP servers, right, for your kinds of use cases that you'd like to do. And so I'd like to spend some time just kinda walking through each of the MCP servers that we have in this repo and kinda talk about the different, skill sets that they cover. So we have five MCP servers that we use. We have our data modeling MCP server. This is one of our, Neo four j labs servers that we have that facilitates, data modeling in the graph. So it'll take your source data, reason over it, and then provide a validated graph data model at the end. We have ingest MCP server, which is built to take CSV files and then load them into Neo four j efficiently. We also have a lexical MCP server, which is responsible for taking PDFs or other types of unstructured files, parsing those, and then loading those into Neo four j in a lexical graph. So this is our, you know, document structure that we wanna load into our graph. And then we have a entity MCP server, which will look at those chunks that we generate with the lexical MCP server and extract, the entities and the relationships, that we've defined back in that data modeling step. And then finally, we have our GraphRag MCP server, and this is what we will be executing our, analytical Cypher queries through, what the agent will be using to do that. And so the data modeling MCP server, again, this is how we're going to validate our data model and then also have a deterministic structure to how we're defining our data model as well. Within the server, we have many different, example data models, across different industries that we commonly see in Neo four j. And there's a whole suite of tools here, but in particular, we're going to be using the validation tool, and then also the mermaid configuration tool. And this is gonna generate a deterministic mermaid style visualization that typically, any agent application is able to then visualize. And we also have a tool that's gonna be able to export this data model to Pydantic. So this is gonna be really important for the, entity extraction steps where we use these pedantic models to validate any return to JSON, from that entity extraction via an LLM. And this is just kinda like the internal workflow that we have specifically for the data modeling. So we'll do some sort of data discovery upfront. Right? So we're gonna look at the sample data. We need to understand what kind of use cases we wanna address. Right? It's really we really stress the importance of having use cases upfront. We don't wanna do, like, an open ended type of data modeling exercise. And then we're also gonna look within the MCP server and see if there's any relevant example data models. Right? Once we have all this information, we'll generate our first draft. We'll do some validation, and this is, again, going on within the MCP server itself. So it's executing, validation functions to make sure that everything is in check. We will visualize the model, and then we, as the user, can provide feedback to the agent. And then it will take those corrections and kinda redo this validation visualization step, or we can move on. And this is Cypher generation, but, really, you know, this is, you know, generating the ingestion Cypher or also generating those pedantic models that we discussed. And so this is kind of that iterative process within a larger iterative process. Right? And so the ultimate outputs from this, we're gonna have two different types of data models that we generate, depending on our source data. So if we're working with CSV files or relational data, then we're going to have our, you know, structured, data graph data model that's gonna be sourced from that. But then we can also have unstructured data where we're kind of picking out the entities and the relationships that exist within this unstructured data corpus, and that will be our entity data model. And both of these can overlap. We actually want them to overlap if we're including them both in the same graph, because this allows our structured data. So the example that we'll show later is, patient journey information. So we can have the patient journey structured data that's contained in CSVs, and then we can connect this to PubMed research that has, entities that have been extracted. And so, for example, we can have medications that are named in the structured data and then medications that are named in that entity graph. And by matching on those medications, we can now connect our patient journey with research in our PubMed articles. And so that was all the the data modeling NCP server. And so the second one that we have is the ingest server. And so this is specifically built to, efficiently loads CSV files into Neo four j. So, this just has a single tool that just facilitates that process, and so it allows us to load, those CSV files in bulk. And with that, I'll have Jean Marc kinda walk through, the unstructured data focused, MCP servers. Yeah. Thank you, Alex. So the third server that we have in the MCP workspace is what we call the lexical graph MCP server. So the idea is to build a lexical graph representation of your PDF. So PDF, it's we can consider it as unstructured data, but they have a a structure. You know, you have titles, subtitles, you have paragraph, you have images with captions, tables, etcetera. And the idea is to find the best way to represent the structure of the PDF directly as a graph. We do that for several purposes because having this structure gives better context to the LLM when we want to do the extraction. It also allows the agents that will use the lexical graph to know where to find the information. It's better also to chunk the the data, the text with structural constraints. So there is a lot of benefits of building a lexical graph. The problem is that PDF is an awful format. It's built not to be machine compatible, it's built for print. And so it's quite difficult to pass properly. There is not one single method that will allow to take a PDF and build a lexical graph from it. So this is why we created several mods. So this is what we you see on the on the soft points. The first one is using pynew PDF, which is a library that extracts the text from the PDF very quickly. So it's good to use this when you have a lot of text in your documents with a very simple structure. Then using this, we'll you will get fast results. And also, well, we use it to extract images from the PDF and also we extract the tables as images. Okay? So this will be safe. So you will have a very simple lexical graph which will be document with the metadata of the documents, the title, etcetera, linked to chunks. And the chunks will be linked so text chunks will be linked to tables or images if they contain this type of of elements. So this is one type of parser. So probably that if you have a complex structure, then you may so for each page, a time new PDF will not get the proper heading order. So for example, if in your page you have text blocks, you have several columns, you have image with caption, etcetera, the new PDF will not be good for that. It will extract some things but not in the right heading order. So if you have this type of documents, we built the doc link parser, which is an open source library for IBM. That uses a machine learning model to analyze the layout and recognize the different elements in the layout of the PDF. So it will create elements. The elements can be title, it can be image, it can be caption, and it will try to get it right for the reading order. K. So here, we automated the so if we use Dockling to automatically build much feature and complex lexical graph. So probably that Dockling will run on your machine and so it's not it will be quite slower than PYU PDF. And it's difficult to parallelize. It will depend on the number of CPUs that you have. But the result should be quite good. And the last one is the page image, and it's what I plan to show in my demo. If you have very complex documents, like, for example, a PowerPoint presentation that has been converted into a PDF, you imagine that you cannot chunk it. So page is the chunk and you cannot just extract the text from it because the position of the text in the slide is very important. You may have tables, columns, etcetera. And so this model, what it does, it creates a simple lexical graph with chunk with document and pages. That's it. But on the page, you get the extract of the image of the page and the extract of the text contained in the page. And we will use LLM with vision to do the extraction using both information, the image and the text. And with that, we get quite good results. And, hopefully, this is what I'm going to show in the demo. And so last model is more experimental. It's called VLM blocks. I wanted to test something between that uses Pylue PDF to extract very quickly the text and that uses the image and the LLM with vision to try to get the reading order. So it's quick. It's a lot faster than the cling, it's experimental. So it may work or it may not work. Yeah. Everything with the lexical graph is a sync because it can take time, and I we try to parallelize everything so that it's fast. And so what we see on the right is the tools. So you have create lexical graph that will create nodes and elements, well, depending on what model you're using. Then if you use duckling and page image well, duckling and Pinyu PDF, you want to check as a lexical graph. You have also tools to complete to complement, for example, duckling. When it extracts titles, it extracts just titles like that, not the substructure of the title. So assign section hierarchy is going to take the titles, send that to the agent and ask it to create the hierarchy based on the name of the title. Okay? If you have one one one one one one, for example, it will create that and create this structure in the graph. Then you have tools to generate chunk description. So for example, when you have table or an image and you want to make it searchable through vector search, one way is to ask a VLM, an LLM with vision to to create a textual description of the image. K? This will use the context if you have caption, etcetera. And then you have one to create embeddings for your chunks, either text chunks or the described image of table chunks, then to make it searchable through vector search. And so last tool here, the can be referred lexical graph. It's just a way so you can try it on one or several documents. It will recreate markdown file from what you have in the graph, one document or several again. So that you can compare what you have in the PDF, what you have in your markdown and see if it needs to be improved and if the reading order, also chunking is correct. Next slide. So this was how to get the first graph from the structure of the graph. And this is an example of a lexical graph. It's not totally complete here, but this is for you to get the idea. So you have a document node that is central, but you have one document node per document that contains information about the metadata of the document. On the left, you can see chunks when used by your PDF, for example, of the page image mode. You you will have documents and chunks that are part of a document. And the next chunk relationship so that when use an agent to find the chunks, he can find the chunk after, before, etcetera. When you use PyNew PDF, as I said, it will extract image and tables that are contained in the text chunks. So the text chunk will contain only the text. And if it contains an image, there will be a link and it will create an as element relationship with the image of the table node. And these nodes will contain the image as base 64 directly saved in and written in Neo4j in the node plus any other data that are created like text that were extracted from the image of the table and the text description of the image of the tables. When you use Dockling, you will have also section nodes. And if you use a tool to assign a hierarchical section, it will create a full section as a tree with a subsection and next section relationships. This will be directly linked to the chunks, so that when you have a chunk, you know exactly where it is in the tree representing your section of the document. Next. Okay, so then when you have a lexical graph, so you have chunks with text, with images, with embeddings. And what we want is to extract a graph from this content, not embedding, just the text and the images. And so here we have the entity graph that exactly does that. So the first step will be to convert the schema that was built using the modeling MCP server into pyodontic extraction model. Why we do that? Is that it's because pyodontic has a richer structure to define exactly so first, it's used to constrain the output of the extraction LLM. Okay? And when we do that, we can define and refine and be very precise about how we want the output of the LLM to be. And so the first step is to convert the schema into a pyelantic structure, and then we will use agent, so CluedCode, Cluxor, etcetera, to based on what you understand about the data, to build what we call the validators, Palantir validators. So for example, a certain property maybe, what you want is that this value will be defined from a finite list of possible values, in case this will be literals. So you don't have to understand all the details here, the agent will know. But you can describe what you think about the model would look like and then test it, see the extraction and then ask the agent to refine it. And why is it important is that if you define very precisely how you want to extract your cuff, you don't have to do a lot of entity deduplication afterwards. For example, anytime, I don't know the country, America will be saved as a node, If you design properly or you want to write names of countries like with a capital letter or with lower cases, with type etcetera, then you make sure that whatever chunk that is talking about America will be linked to the same node. So the entity graph, it uses LLM to do the extraction and it's automated if the chunk contain only text, it will use LLM. And if so only the text and context will be sent to the LLM for extraction. But if it contains an image, then it will use the image and the extracted text to do the extraction. Here, we use the same type of model. So we're going to show in the demos that we use GPT five mini, and it's good to use one LLM to do both type of extraction. This part can be slow, especially if you have a lot of chunks, but it's totally paralyzed. And so if you use OpenAI or end user type of LLM, It will depend on, I mean, the constraints that you have in term of rate limit. But here in the demo, we can use 50 call at the same time, and so we can extract 50 chunks at the same time. So it it can be quite fast for text. And for vision, it's a bit slower, but still it's manageable. Next, and the last server that we have is the GraphQL server. So once you have a lexical graph, a stricture graph, an entity graph, and all linked together, this MCP server gives the ability to the agent the coding agent to look and search inside the database. So it has several tools to do that. You can use PureCipher to search and even to so to to read, but also to write. So if you want in the full process to do some preprocessing, post processing on your graph, you can directly ask the agent and you will be able to write anything in your database or delete things in your database. So it can read, but it can also use vector search and full text search. So generating some reading is done outside of the server, and so the agent just need to, well, call a tool and it will do a vector search automatically. And then there is a tool to do everything at the same time. If the agent wants, it's possible to do full text search plus vector search plus a graph traversal in one query. So this is a search type of query. If from my experience, the agents are lazy. And so they will not use this tool because they need to think a bit more about what type of queries they want to build. But if you ask it to use it, they will build very nice queries. So in one query, they will be able to start from a vector and full text and build a query. So it's quite nice. And so last tool is that gives the ability to the agents, the coding agents that you will use to read also the images that are stored in Neo four g in the graph. So that's quite nice. And so so to answer some of time of very difficult questions that needs a visual representation of your table, your images that you have expected, then the agent will be able to use that. So this server is used at several parts of the workflow because this is also a junk at check that all the steps are coherent and that, for example, when you build a lexical graph, you have the right number of documents in the database. Next. And, yeah, I let Alex continue with the skills. Yep. So, yeah, so we're gonna talk about, some of the skills, that we use to orchestrate orchestrate these workflows that we've defined here. And so we basically have two primary modes of function that we're defining with the And for those of you that aren't familiar with what a skill is in, like, the AI context, it's basically a a markdown file that just defines how a particular task should be done. And within the skill, you can reference external files as well. And so this allows the agent to kinda selectively expose context into the window. So we're not putting every single skill within this MCP workspace into the context window all at once, but the agent is able to identify, okay. If I need to handle unstructured data, for example, now I can pull in this subskill that tells me how do I handle unstructured data in the context of this workspace. And so it's just just a better way to manage your your agent context and then also provide, like, specific details on how to do particular tasks. And so in this case, we have two primary modes. So we have a analytical mode and then a chatbot mode, with the main difference being that the analytical mode is going to be more tailored for handling, like, use cases. And then the chatbot mode is gonna be more for, obviously, like, chatbot implementations, but, like, handling, like, question and answer type tasks. And so both of these follow a very similar structure that's defined within the skills. So we're gonna start with data discovery. So this is gonna be, you know, reading our source data, our tables, our CSVs, our PDFs, and then generating, some use cases or some questions that we can then use to evaluate how well our, graph is at the end of this process. And then once these are defined respectively, then we have our data modeling stage. We then use the respective MCP server to ingest our data according to the data model that we've defined here. We then verify the ingestion. So there's just some simple data validation checks that the agent can do, such as looking for, orphan nodes or making sure that relationship and node counts are what we would expect to be, like, within a given range. And then once that's done, if we have some PDF information, then we go through that entity extraction step, that Jean Marc walked through. And then, finally, that should mean that we have our entire graph, completed. And so then we are going to write Cypher that is based on these use cases and our data model and execute that against the graph. And we're gonna come up with a report that basically is saying how well our use cases are addressed by our current graph. And then we have an opportunity to go back, right, to maybe the data modeling step, or we can even provide additional feedback in the Cypher queries that we're using, and the agent can then make corrections as seen as we see fit in order to better address our use cases. And so, really, we just want to reiterate here that this workspace, there's quite a lot going on with it, but, really, it's just a set of MCP servers and then skills. And by combining these, we can get really powerful workflows, as the end result. And so we've kinda talked about this throughout, but we've tested this primarily with quad code and cursor and then using OpenAI, LLMs, and embedding models. But even though we've only tested with this, we believe that since this is really just MCP servers and skills, this should be easily portable to other agentic systems as well. So, you know, this should be usable in applications such as, like, quad desktop or other agentic IDEs, such as, like, OpenAI Codecs. But, you know, if you're using one of those systems and you find some issues with implementation, we'd love to hear that feedback. So we're gonna share a link with you to the GitHub page so you can feel free to raise an issue, and we'll take a look. Because our goal is to make this easily accessible and make it able to integrate with Popular and other applications out there. Maybe let me add something. Yeah. So we have the coding agent. So that usually it's Sonnet by default in CodeCode or Cursor. I mean this is what we use the most. And so if you want to use full open source, there are some so client and OpenCode, for example, are open source alternative to ClotCode and Cursor. And you can use OpenSource LLM inside. Usually, I mean, for recording assistant, you need quite a large LLM to to I mean, if you want to do something complex, let's say. But because of the skill, it is quite described and using the MCP server is quite easy. It should be able to work with a local LLM. We haven't tried. But the second thing is that to build the graph, we use also for the lexical graph and especially for the entity graph, We use GPT five minutei by default. But I tested it with open source LLM using OLAMA. So first, all the servers use light LLM. So it should work directly with under plus type of LLMs and LLM providers. So if you want to try that, read the read me for each server, but it should be easy. I tested with some. And especially, I tested the extraction with was it FIFA from Microsoft, which is a very small model? It did some extraction. The results was not as good as GPT five mini, but it did some extraction that was correct. I also used QUEN 2.58 bits 8,000,000,000 parameters, I think. And it was doing some great extraction. So you need a lot of GPU or to to have more RAM on your Mac if you're using that, but it was quite good. So probably is that it's not parallelized, so each chunk is going to do a LLM call and so it will be slower. And so last thing is that I tried also with Quinn three nine billion parameter for because it has vision capabilities, and it was doing some great job as well to extract a graph from images of tables plus the text. Okay? So you you have open source alternative for everything. It's just that, yeah, we haven't tested everything. Also possible configuration as possible. Yeah. Sorry. Shall I go on the Alex? No. That was good. So yeah. So and, again, really so what this is at the end of the day, this is just a workspace. It's hosted up on GitHub. You can just clone it down into whatever environment you wanna work in, and then we'll kinda, like, show you how to set it up rather quickly. And, you know, it's just MCP servers and skills. To be very clear, this is a tool that Jean Marc and I have developed over the past, few months or weeks. It's it's not a Neo four j product. This isn't in Neo four j labs yet either. So this is just something that we've been using, out in the field, to do, you know, POC development or exploration of different graph use cases. So the results that you get, you know, it's pretty fast. It's powerful. It's really it can get you great value, but we still need to be building production systems kind of on top of what we learn and on top of the data models that are generated, on top of the Cypher queries that are generated from this workspace. We can take that and then modify it and bring it into our production workflows. And so with that, we're gonna get into the demos. I don't know. Jean Marc, do you think we can just go into the demos now, or should we pause maybe for questions just to clarify anything? That's a good question. Again, yes, I saw some questions about, for example, adding support for Bedrock. So it should already work with Bedrock. So again and LLMs again, there is different things here. There is coding agent. So it's code code or cursor. And if you want, there are some ways to use code code and cursor with a Bedrock or a user LLM inside. And there is a extraction model and embedding model. And this should be out of the box compatible with Bedrock. I haven't tested it, but it should work again because we use the light LLM, which is a library that allows to well, to to be compatible with a 100 plus LLMs and embedding models. So it should work out of the box. And the the way to use it is that so by default, the name is OpenAI for the environment. But if you put your bedrock key and you say which type of model, you start with bedrock slash and the name of the model, it should work out out of the box. Yeah. I do you see any other questions you want to answer, Alex? I could not read everything. I tried to answer directly in the chat. I think, yeah, with the demo, will be more concrete and maybe we can ask questions directly after the demo. Yeah. Yeah. Definitely. We we had a we had a lot of lot of questions. Obviously, I see some of them in the q and a tab. Maybe that's a little bit easier for you to look at, Jean Marc, not in chat, but the q and a if you filter by un unanswered. But, yeah, I I I think maybe the demo clears clears a couple of things up. I I think considering it some questions are going in that direction, how to do, what to do. Can you give an example and sort of that? So I think we we can clear some some questions up with, hopefully, with the demo, and and then we can dive into questions afterwards. I just want to address one thing. I see one of the. it's not the question, yes, it was saying Light LLM has been compromised. So yes, yesterday, yes, we saw that there was a big thing about Light LLM because the last version was totally compromised and it was not I mean, it's not even possible to get them from PiPi. But I changed the code this morning, and so it use a safe version of Light TeleM. So do not worry. It should be okay, and it works, and it should not be compromised. Great. Yeah. So with that, we'll do our first demo. So here is the link to the the workspace. So this is in the Neo four j field GitHub organization. It's a public repo. And then if you would like to see the data that we're using, you can follow this link as well. And maybe it's good can we is it easier to post these in the chat maybe just so they're more accessible? Yes, And then do it. I yeah. Then once I move off this slide, people can still get these. Yeah. So Alex shared in the chat already the GitHub repo. I'm going to paste also. So this is a Google Drive. So the tiny URL link, it contains three folders. One contains the slides that we are showing and the video of the demos, so in case anything go wrong. And we also have the data the two demos. So, yeah, if you want to do it at the same time for the Alex demo, yeah, use the data the demo one folder, and you will see the PDF and CSS. Cool. Alright. So before I show the demo, just wanted to cover some prerequisites. So we would need, Neo four j, with APOC installed. APOC is a library plug in that just allows for some additional, functions within Cypher. And then we need, UV. This is the package manager that we use, for our workspace. Git, obviously, your own data, as we said, cursor or a plot code, or, again, feel free to experiment. Please give us your feedback if you're using anything else. And then, we do need an API key for OpenAI to use the entity extraction models, which I believe we are providing as well. Is that also in the chat, Alexander? Maybe. Maybe. I I see the link from your market. See the GitHub repo. Which other link should we add? With Tora? Just the API key for OpenAI. so. that yeah. I'll I'll put that in I'll put that in chat right now. Yes. Yeah. Cool. Yeah. And so we'll provide this API key, and then so we can use it during the session, and then we're gonna, close it out, at the end of the day. But feel free to experiment with it in the meantime. And then once you install the repo, there's gonna be setup instructions in the read me. But, basically, we just need to run this setup script, and then this is going to handle, you know, setting your environment file and installing your MCP servers and making sure that they're, functioning. And then we can run this, setup workspace skill, and this is going to ask whatever agent that we're using to then confirm that everything is up and running and available as we would expect. So pretty simple. Just run this, setup shell script, and then once we get into the, agent's window, we can run this, setup workspace command. And so I think I got one more slide here. Yeah. So the demo that I'm gonna be going over, is with, patient journey data and PubMed articles. And so some use cases that we'd like to address with this data. So, for example, we'd like to say, we wanna link, a particular patient with specific research from PubMed that supports, medication and treatment decisions. So really, again, going back to directly connecting patients to the pertinent research. And so we're going to have a collection of CSV files and then some PDFs. So this is gonna do a mix of structured and unstructured data. And so our full graph data model is going to look something like this. This is not the exact data model that we're gonna use, but this is kinda just to get the point across. So we're gonna have our lexical component up here that's going to have our document structure. Then we're going to have our domain component here, which is gonna contain our entity graph in green and then our patient journey graph in this tan color. And so each of the entity nodes is going to have a relationship to their respective chunk node, And then we can see that there's some overlap here in the patient journey and entity components. So, in this data model instance, we see medications, clinical outcomes, and medical conditions exist in both the entities that we're pulling out from the PDFs and within our actual patient journey data. And so because these overlap, we can now connect our patient instances with, chunks in our, lexical graph. So let's see. I'm gonna go to the video here. And so this is just gonna walk through, the end to end process. So starting with running that shell script and then going through the processes that we've defined here and ultimately, doing some analysis on our resulting graph. So I'll go ahead and play this, and we can just kinda, like, talk as this is happening. Let's see. There we go. Okay. Yeah. So we're gonna set up set up our repo, and so we should get something that looks like this where we'll get a set of complete notification once we input all of our information. And then this is in cursor, obviously, but then we just need to open a window, and we run this setup workspace command. And then we're gonna go through the process of validating that our MCP servers are installed, that we have our environment file configured, that, MCP servers are actually running, and then we can access the tools. So, yeah, we get this nice little output here, and everything checks out. And so then here, I have a file that just has this, prompt that I like to use, for, testing this dataset. And so it explains where the data is and then also describes the use cases that I would like to address with the resulting graph. And so we run this develop Neo four j graph command, and then I just tell it to reference this file. And so here, it's going through that discovery step. It notices that the the data, directory is empty. That's not true. It's actually just filtering that out because of a dot git ignore, I believe. So we have to redirect it, but eventually, it finds the data, and then it's able to actually start doing some analysis. And so we see that it's reading the PDFs that we have available, and then it's reading in the CSVs. And so here, we can see kind of that that structure that we saw in the slides where we have the that step process that it takes. And so here, we're doing the first two steps of discovery and then, determining the use cases in the mode that we're going to be in throughout this process. Yeah. And so it's just doing some analysis here of our CSV data, making sure that the IDs are in sync across all these files and that they're usable. You can see it was up there for a little bit, but it noticed that it had both structured and unstructured data. And so it was able to look in this references subdirectory for those markdown files that describe how do I handle structured and unstructured data. So here, we're doing some more analysis. Alright. So we're gonna do our final report here, of our discovery stage, and we've decided that we're going to be in analytical mode based on that initial prompt that I provided. And so now we're going to generate our graph data models that we're going to use. So we're going to create a graph data model for our, patient journey data based on the CSPs, and then we're going to create our entity data model to actually extract information from our documents. And here, we're just seeing, okay, we have a patient journey example, within the MCP server, so we're gonna use this to base our new data model off of. And I see ROC is saying that this is a little blurry. I hope is this blurry to anyone else, the screen share? No, it's okay. I can read. Okay. So here, we're just generating the data model. So behind the scenes, it's generating the JSON version of the data model to then, validate. Alright. So we ran this get mermaid config string. So under the hood, this actually also validates the data model. And so this is described in the skill. It tells it, you know, if if you know that you need to visualize the data model, you can just run this one tool instead of running the visualize tool and then the oh, sorry. The validation tool and then the visualization tool. So just a way to reduce the amount of work that we're doing. Just to comment maybe. Yep. So here you see that because of the skills, everything is more or less automated. But at any point, you can ask it to explain what it is doing, why it is doing or propose something else to do. So I think it's a very good learning exercise to understand how we can build a graph for all the MCP servers are working. And so it gives a lot of flexibility. So I use them all the time because it's fast and it's prewritten code, but it gives a lot of flexibility as well. So don't hesitate to stop and ask questions to the coding agent and and propose all the solutions and our or ask it to propose more solutions to to your problem. Yeah. And, usually, it'll actually stop after this data model generation and ask you to validate that the models generated are correct. Here, I decided to just go right into ingestion. So I think I'm actually yeah. So I'm gonna stop it here and ask them to review the data models. Yep. And so now it's gonna visualize this for me. So, you can ignore, the the patient ID, and it looks like it included some properties up top here. But, basically, these models are what we're looking for. So we have clinical trials, medications, drug classes, conditions, and outcomes. And up here, we have a patient journey representation. It looks like we have conditions and medications there as well. And so, yeah, so here it's giving us the unified model. So we can see how these two are structured data, patient journey model, directly connects to our entity model. And so this looks good. I'm just reviewing a few more things here, and then, eventually, I'll give it the go ahead so we can start actually loading our data into Neo four j. And because you have the question, so if you want to modify the data model, you can just interact with agents again and propose a different data model and it will update it based on what you're saying. And you can note that the data model itself will be saved as a JSON file and then a python tick Python file. So what is used to to import the data to ingest and to do the extraction is on the file. So you can even edit manually the files if you want to do it. Yeah. Yeah. Definitely. Yeah. All the important resources are exported, so we can just persist all that information, which is great. And so we I see a question. Do you create ontologies or ontology? The answer is, well, we produce and we use not ontologies like what you in the RDF world, but directly graph scale. Why? Because it's more compact and especially when I talk about the extraction, it's more compact and it's directly taking into account. I mean we want to constrain how the LLM will output its result for each chunk. So it's a very good way to make sure that the LLM is not going to hallucinate new node types or new relationship. It's very constrained and especially GPT-five minutei. It cannot use any things that is out of the structures that we define. So this type of model, you can consider it's Graph Schema, and it will be the equivalent of the ontology, but we don't use a release ontology. If you want, you can import ontology and ask it to convert that to Graskema and it should do it automatically. And so where's the question? Where is the agent here? Do you want to answer this one, Alex? Yeah. So this is yeah. This is using cursor. So the agents is we're using the agents mode of cursor. There's a couple other modes. I believe they have, like, planning and a few other ones, but this is just agent. And then we're using Sonnet four dot six, so you can there's a whole bunch of different agents that you can select in this drop down. So that's where you configure the the actual LLM that's doing the work here. And then we've you know, in that setup scripts, we downloaded and configured our MCP servers. So this is a video, but, basically, if you if you go into your cursor settings, you can see your MCP configuration is right there as well. You can see what's running and, what's broken. And then the the way that cursor works is that it will we actually let's see what's here. Yeah. So we used to have a, an independent, dot cursor, directory, and this is where we kept cursor specific skills and whatnot. But cursor actually reads from dot cloud now. So we migrated everything into dot cloud, and so this is where the skills are currently contained. And so the agents just automatically pick up this information. So it has direct access to the the skill sets that we define here. And because we have other questions about the agent, tell me when I have to stop explaining or, answering. Sorry. the question. Do you want to explain what's happening U. Or should I address all the questions? Yeah. Maybe just an update on what's happening here. So we've ingested our, structured data, and so now we're doing our PDF parsing. So this is a lexical graph. So as we saw before, so it will take the PDF, extract using Pinyu PDF the text and create chunks and and documents and link them through relationships. Yeah. Everything again is parallelized here. And so it's a job that is running and you can check the job and you can cancel it. Yeah. If you want to know what's happening. And so going back to the agent, so here basically Cursor or CluedCode, it's an agent, an LLM that has access to tools that it can use to code an application or to create a graph for example. This is the tools are in the MCP servers. And so this is what the so what we're showing here is a way to automate, but that gives a lot of flexibility. So the agent have a set of tools. It knows what to do through the skills, but it can deviate. And in the skill, you say, analyze the the documents, for example, and propose the best schema. So there is really some kind of intelligence and well, and using the tools, this is how we are able to automate the full process, but also to make it very flexible. So again, if you chat with the agent and you give better recommendation about what you want to do, it can do it. I see other questions here. Will it work in GitHub Copilot? We haven't tested it. I think GitHub Copilot is can use MCP server, so it should work. Skills also I mean, MCP servers and skills are really stand out now. So that's that's just that. So maybe it needs to be installed manually. But, yeah, you have to go for the MCP server. You have the skill. You just need to put that in a in a GitHub Copilot. Yeah. Do you want to. explain what's on the screen? Yeah. So here let's see. So we've ingested our, lexical graph. And so now this is looking at the entities that we've defined in our entity, graph data model, and these are the pedantic class or the pedantic models, of those entities and their relationships that we wanna pull out of our documents. And so here, it's looking through, and it is modifying the Tideantic validators. So, basically, what's going on here is for the properties that are strings, we want to strip the white space from them and then also cast them to all lowercase. And this is going to make entity resolution, much easier, not only for the entities that we're pulling out, but also when we go to ingest these entities, we wanna make sure that, like, a medication if a medication exists in the patient journey graph, we wanna make sure that we're matching on that same instant instance that's already in the graph. So this is just helping that process, be more, complete. Yeah. And here, based on your data, it will not always use a strip and lower. It should or you can tell it if you know your data and you know how you want to format the properties in the nodes or in the relationships, you can use very complete constraints here on the output of the LLM model. Yep. It's all like like type of, yeah, constraints or validation. Yeah. That entity's pretty powerful. So you you can do a lot of advanced validation, to, like, massage your extracted entities. And so here, we're actually pulling them out. So we see we have 275 chunks. And so now we're gonna take that all those pedantic models that we just defined, and we're going to use those to define our structured output from our LLN calls. So you see here you have two fifty text chunk and 25 I think it's table chunks. And so this 25 table chunks use LLM with vision to do the extraction. So it's doing it. Yeah. Let's see. Alright. Here, I'm kinda speeding ahead here. So Yes. There's a few lost. can take time. Again, it's parallelized. It depends on your OpenAI or end user subscription. And especially the VLM, it can be a bit slower. Yeah. And I think in this case, I think the VLM was taking a little bit. So I think I just told it to continue without extracting the last couple chunks. Building the lexicograph or the entity graph are processes that are run-in the background, you can still talk to the agent and tell it to do something else. So if you want to continue, you can do it. Yep. And so, yeah, once we get those ingested, then we're we see that's executing the read Neo four j Cypher tool. So this is just validating, that we extracted the entities that we wanted to, and it'll give us a summary of what that looks like after it's done. Yeah. So that's just an example of the results set that we get from one of the queries that it was running. So it's just doing some more checks. Looked like, briefly, we saw one of its thoughts that said that there might have been an issue with, one of the bridge nodes just in how we were naming a medication. Yeah. It looks like type two diabetes. We say type two diabetes a few different ways, but now it's actually gonna go in and fix this in the graph for us. So this right Cypher query tool allows it to execute right queries, and so it can fix any sort of it it can deduplicate entities and nodes in our graph for us automatically. And so that's what it did here with the type two diabetes medication nodes. And so now that we have all of our data in the graph, we're gonna go back to those use cases that we defined, upfront, and now we're going to start writing analytical queries to actually address those use cases. And so this is the agent executing, the Cypher, and we can see that was just one of the result sets. It looks good. I see a black screen. I don't know. So the screen turned black. So some of these things that might need a vector search. Yeah. Something is wrong with the video. And then once it decides that these are satisfactory, then it's gonna create a YAML file that actually just contains all of these queries and their question pairs for us. I, I think you cut. yeah. The video is is no longer sharing. Maybe you do you wanna start and stop? You just see a black screen here. Yeah, I think he has connection issue. Maybe maybe Alex is we? have lost Alex. Oh, Yeah you're frozen. here. You're frozen. Yeah. We don't see the, yeah, the video anymore. It's frozen again. It's not moving. Yeah. So I guess he was going to show the graph maybe. Yeah. You're back, Alex. It's. back. We can. Okay. see your screen. Sorry about that. Yeah. No worries. It's back. Okay. Okay. I think this is something else I was working on. So let me just bring this up. Sorry about that. Think we were somewhere around here where we were executing the vector search. Yeah. So here, we're creating that YAML file with the queries. Should I answer some questions? So I saw one question. What is the difference between unstructured.io and your PDF model as you showed? Basically, our competitors are used on the structured dockling and there is a Chinese one also that I use. They are both of the strengths and weaknesses. And so depending on the the type of PDF, I would use one or the other. I implemented docking first, but, yes, the idea is to implement unstructured and the other one as well. Maybe another question, can you explain why Anthropeak or OpenAI won't make this work obsolete within the next few iteration of our model? Well, we we are Neo four g. I think I don't think they have OpenAI or Anthropique have any plans to build a graph database. But if in the next iteration, create a tool that does not need what we're building, I would be quite happy if they can do the same and build it directly in GPT or include, I will be quite happy. So I hope I don't think it will be the case soon. Maybe we'll use different format. Now it's MCP server. Last year, it was different. But the main idea is for well, we built that to allow our users to quickly generate a graph of quality to show the value of the graph and to understand all the different parts in prototyping this graph and explaining and iterating to make sure that when you go to production, you already understand what you want to to build. Yeah. And just to to catch folks up on what is going on on screen, I asked it to, maybe enhance one of these queries that we had, one of the analytical queries. And yeah. So here, I wanted to match a patient ID specifically and then go find the studies that are directly connected, traversing through, the medication nodes that they're taking. There is a question, why are knowledge graph important in building intelligent applications? Do you show the graph at some point, Alex? I I don't have it in the video, but. I can show it. I can just pull it. up here because this is the local graph. No. So so to answer the question why knowledge graph are important in building intelligence application is what I was trying to explain in the introduction is that your data, it can be tables, it can be unstructured text that you want to check. But you have ways to store this data in relational tables or in vector databases. And the thing is that they are all stored as independent and not connected structure. So basically, an agent can go and search this type of databases, but there will always be independent pieces of information. While in Neo4j or in graph, what we do is we connect all these data through the relationships that are first class citizen. And so you can imagine that you have information, piece of information, but when you connect the pieces of information, this is when you build the knowledge. This is exactly what happens in the brain. You may have different pieces of information. But when you connect this information together, this is when you get the knowledge. And so when you provide this type of graph to an agent, it becomes a lot more clever and it can understand I mean, it can answer better type of questions. You have also use cases for graph for agent, for example, what we see a lot is using graph for the memory of the agent. So maybe we can share some blog posts or demos in the chat or building the context graph to well, so that we can automate any type of processes. Yeah. Do you want to show something, Alex? Yeah. Yeah. So that is the video. But, yeah, I do have the graph up here as well, so I can just quickly show the resulting graph. So this is that last query that we ran. So looking at the patient and then trying to figure out, what research is directly tied to it. And so we can just run that. And if we see we get Joe Smith, get some medication information, and then we get, just a preview of this top chunk that we can easily configure this, right, to return, you know, maybe just we're just interested, right, and maybe just the the context of that. And we can see so we can just do a quick view of this. Yeah. So we can see here. So we have a condition, and we can see I can zoom in here a bit more. Okay. Well, here we still have a patient. We can see the journey events, so enrollments, the identified care gap, things like that. We see claims that they filed, and then here we can see, condition that they have. So type two diabetes. Looks like type two diabetes is pretty common in this graph. And then we can maybe see what else is going on here. So in this type two diabetes node, we can see all the text chunks in this pinkish purple color, that it is extracted from. And then we can also see so that's interesting. So it was extracted from a table as well, we can see actual raw table results that, were also from those PDF documents, and then we can see, medications and clinical outcomes that are also associated with type two diabetes in our graph. So that's, that's all I have for my demo. Are there any other questions in the chat here, Jean Marc, that are related to to this one? saw one for you. Can we keep the actual data in the relational database and utilize new four gs just for the knowledge graph? In a way yeah. And there's different ways to do that. So, one of this is kinda getting into other applications, but, a common use case that we see is using Neo four j as the semantic layer, for, like, relational data source. And so we can basically take the structure of tables and columns and metrics and all of the metadata associated with it in, relational systems. That's easily mappable as a graph, and so we do that in Neo four j. And then we can do, semantic similarity and graph traversal. So combined, retrieval strategies in Neo four j to get sub schemas and then generate SQL from that sub schema in Neo four j, to could then query our relational store, whichever relational store has been identified as, containing the right information for a particular task. So maybe we reset. I see several questions about the embeddings. So here we have two different type of data. We have CSVs and we have PDF. So the CSVs are structured, they are imported. There were several maybe dozen of CSVs And automatically, they were joined as graph imported as a graph. And then you have the PDF. So the PDF, they were chunked using PYMU PDF. And each chunk, I don't know if you did that actually. It's an optional step. So each chunk contains a part of the text. And there is a step where you can create embeddings of the chunks. So using the embedding models that is defined in the config file. So then, yes, I think you did it because after you used VictorSearch, so the agent, the coding agent can use a tool to search the database through Cypher queries, but also through Victor search. So he's able to find the chunks that talks about certain thing. Yep. Let's see. I think I saw we see? about entity resolution. I didn't see. Okay. Well, if I find it, maybe I'll answer in the chat. But, do you wanna show your demo, Jean Marc? I can do. you think? So we have that, and I think we have the future of the work. Maybe I can start. Oh, yeah. Yeah. Just and and cut me ten minutes before the end. So I would just show the beginning. I will try to be quick, but just to show a different ways to to use this. And so do you think ten minutes would be okay for the end last slides? Or maybe five. Yeah. Yeah. I think we just need, like, maybe five or so minutes to go over the the last few slides here. Okay. Let me share my screen. Here I have another type of demo, so I'm going to use CrootCadr. I'm going to do it live, I won't be able to speed up the video and get some results. But I want to show you different ways to use this. The idea is that I have four or five PDFs that are complex that are coming from PowerPoint presentations. And it's three PDF coming from pharma company who are presenting their results to investors. So this is public PDF. And the idea is what how can we use that to build a graph and answer complex questions. So this is what I'm going to show. And in this example, it will be like a competitive intelligence. So you can imagine that we work with pharma companies, and they want to understand what their competitors are doing and maybe to do some collaboration on certain type of drugs, So this is the idea. I will start from scratch. So here, I have so first, let me take the repo. The repo is here. And I can just git clone into my terminal. And so it's cloning, and then I can just go inside. Okay. And I have a setup script that I can run. I will use my local database. I will use so here I need to put my credentials, basically. So Neo four g, my password. My database is Neo four g, and it will use my OpenAI API key. Let me copy paste it here. So embedding model, will use the default one, which is TextUnbedding three small GPT five minutei. And then it asks if I want to install Dockling. The answer is no. Dockling is quite large, So it needs to install PyTorch and and download the model. So not for this demo. Then it's checking on the workspace. What I can show is so I use generally my terminal like that and cursor because I like, well, to to visualize what's happening. So Alex already showed us that the skill is here, and we downloaded the code for four of the MCP servers here. So now everything is ready. I can start cloud, and it finds five MCP server. That's fine. So let me check something. In the code code, if I want to check my MCP servers, I just do that, and I see they are still connecting. Let's wait a little bit. And the only thing that I will need to do is copy my PDF here. So let me take my PDF. Okay, and I can show you the type of slides we are going to work with. So this one is from AbbVie for example. You see the type of slides and we want to extract information, a graph about this. So usually it would be molecule. We want to know who is developing which molecule. So here, AbbVie is developing these molecules, what these molecules are about and in which phases they are. So you see that directly you cannot use, you cannot just extract the text and send the text to an element here as a so layout is very important. Okay. You have colors. You have different columns. And you this slide, for example, is quite dense. But well, this is a slide that I use for test. I know that there is 52 molecules here and with one pass, I'm able to extract all the molecules properly. Another example is that you have this type of slider. So you see each slide, each presentation is quite different. This one is from BMY. And you see that you have name of molecule in text, but also sometimes it's in logo. And same, this is a slide that I use to check that the extraction is correct. We have six molecules here. Yep. Okay. Let me check my MCP server. They are all connected. Great. And so I can start by saying, okay. Let's build a graph from my pdf. And so here, it's a conversion conversational agent. Well and it started to well, first, it's checking the the the that everything is connected. Okay? Then it so the agent, because it knows that we want to talk about graph, it knows that it has a skill describing the methodologies and it looked at the skill and so it's starting the skill. So it found four documents in my PDF folder. And yeah. It's sampling the PDF. So basically, it will look at usually two the first two pages of each document, and it will provide me with a table with a summary. So we have four documents. We have from four company. It knows the dates because it access the first page of each document and the content. You can work with a lot more documents of course. It already saw some content. So you see that we have companies, we have drugs, we have development phase, therapeutic areas, oncology, cardiovascular, etcetera, disease indication, etcetera. And so with the skill, it already understands that this is a complex document and the model that it should use this page image. So it's able to understand which parcel it should use based on the type of document. Okay? Then it ask for a use case discussion. Okay? It ask, do you want to ask natural language questions or do you want to surface patterns? So what I want to say is I want to build a competitive intelligence chatbot. Okay? So now it understands that we want to build a chatbot, and so it goes to a different model that than Alex was showing just in the previous demo. Mod chatbot. It's obvious. Okay. So it proposed a plan. This is totally editable. If you want, you can say, okay, add a step or remove one step. Here it went directly to step three. So it's checking the so exactly what Alex showed already, is using the modeling MCP server and checking if there is already in the resources, data models that would correspond to this type of data to get inspired. It's not the case. And so here, it's proposing to directly build the model. So it's not following the skill. Please propose some questions, some user questions first. So basically one recommended approach is when you want to build a graph, the graph will be designed to answer certain type of questions. And so in the skill, it should propose some questions that this type of graph or the document will answer. So for example, what drugs does a company have in phase three? Or which drugs are currently under regulatory submission or review? Or you're proposing some questions like that. Let's say I want four or five simple questions. Okay. So now it's recommending other. What drugs does company have in Phase III? Which companies are targeting some indication? What is a drug's mechanism of action? Which therapeutic area? And what are we currently under regulatory submission? I don't I don't want the last question. And please fill the blanks. We will use these questions for evaluation. Okay, so this is what I want to show is also how we can iterate and use the agent to better design what we want to do. Okay, so now what drugs does AbbVie have in Phase III, which companies are targeting for their concern? What is mechanism of action? And which therapeutic area as the most pipeline asset? Okay, sounds good. Please use 50 when possible. Okay. So I just say, okay, let's go to next step. And I said, yeah, when you need, please use full parallelization so that we have quick results. Okay. So now it's designed in the graph model. So you see it will create nodes for company that develop and the relationship develops. So a company develops a drug, a drug targets an indication, a drug belongs to a therapeutic area and an indication is part of a therapeutic area. Okay, great. We're missing some the clinical phases here or also indications, but that's fine. So now it's testing. So based on this schema, so company developed drug, drug targets indication, drug being therapeutic area. It's checking that it can answer some of the questions. Oh, I see. So the phase, he wants to put the phase in the property of the Dragonode. Okay. Okay. It should work. Okay. Let's continue because I just have five minute left. But basically, here you could iterate and well, I'm just showing it in terminal, but you can copy paste or you can use cursor to see and visualize as a the model. Okay? Then so it's creating it's ingesting Okay, so here I mean, so you see it creates the document. So it's only created two, now three. And the document have pages and the pages have link between them. Okay? In each page, automatically, this is what I was saying, you have the image of the page. You cannot read it directly in browser here, but you have the text extracted for the image and you have other type of information like the document name, etc. Okay, great. Now it's okay, we have the lexical graph. Okay, so the first document are 61 pages, the second one seven pages, Bayer nine, etcetera, etcetera. So now automatically it will create chunk description. And here I'm using a parallelism of 50. So let me check. So what is sent to the LLM with Vision is the image of the page plus the text extracted from the page and it asks to generate a text description. So you see the result. This one is a single slide, single colon corporate presentation page titled AbbVie long term guidance, etcetera. And you have a full description of everything that is inside the slide. And from my experience, this is a good way to so this is what you need to embed. So to create embeddings so that the agent can search using semantic search this specific page. Okay? Okay. So it's still generating that. Let's check the number of chunks. It's done. Okay. We generated 95 description. And now it's creating the embeddings for each chunk. If I go back here, so you have the text and now you have these embeddings that is vectorized representation of the text description. So automatically it creates a vector index corresponding to these vector embeddings and it creates a full text index based on the extracted text. Okay. Now it will check that everything is good in the database. Okay, so it's looking at the number of documents, checking that everything is good. So we have four documents, 95 pages, 95 chunks, one per page. And this is what is expected for this type of mode. Okay? Now this is where so it takes let me get back to my cursor. So you will find outputs in the output folder. So this is the model that was created automatically. So you have company that will have a name property, which is a string property. You have a drug that will contain a name, a compound code, a mechanism of action, direct type of phase, etcetera, etcetera. And here you have a description of the relationship. And now it uses that to create a pyodontic schema in this file. So you see you have same, you have a company node with a node label company and a name. But it created this field validators, which is a way to output or to define exactly how we want to format the outputs. Okay? And And so it's proposing to normalize everything basically to go to lower case. It proposes to have a description for the phase. But here I can say no, the phase, we only have phase one, phase two, phase three. So I want the phase values to come from a finite list of possible values. Okay? And then you can interact and say, oh, okay, I know that I don't have an infinite possibility for phase. And so, yeah, it's editing the file and creating what we call literal, for example, for the validation. Okay? Hey, And so Marc. when, yeah. sorry. I just wanna make sure that we have enough time for the future work, but we have about seven minutes left. Yep. Let me just show the last one and yes. So it exports the schema. So you see it's right here. And then you can start extracting. Okay. I just want to show the okay, run. And maybe I will show the result of the extraction after if we have time, but maybe we won't. But basically you can do it by yourself. It will extract from the slides that I show all the molecules, clinical trials, the documents, the companies, etcetera. And then it will test on the questions that we have defined at the beginning of the the skill. Great. Should I share my presentation then? Yeah. I have it up too if you need to pull it up. Yeah. I have too many screens open. I cannot. find the Yeah. You can just mute share it. That's fine. no. Okay. Cool. So, so we just showed you kind of the current state of this workspace, but we do have some plans, for future versions of this. So, basically, we want to just continue evaluation. So we've had obviously, John Mark and I have tested this quite a bit. We passed it around to the rest of some folks in the Neo four j field organization, but we'd love to hear more feedback. Right? We wanna make this tailored for how people wanna use it and make it actually beneficial to everybody out there. So there's this evaluate pipeline skill that you can use, much like we use the developed Neo four j graph skill or evaluate setup skill that we were showing, and that will create a, kind of a report file that just kept captures anything that kinda went wrong in the workflow or recommendations, by the LLM on what to fix in the future. And so it's as easy as just capturing that, and then you can send it to us either in a GitHub issue or however you see fit to communicate that information. And we'll use that to improve the MCP servers and the skills that we have. And then on the data modeling side, we want to simplify the tooling. Right now, there's a lot tools that are available, so we wanna compact this and make it more accessible and not take so much context. And then just increasing the compatibility with the MCP workspace, in terms of the Cypher generation and then also, improving how the pedantic models are output to make them more usable by the rest of the workspace, and then also improving the method of sampling, the CSVs and, PDFs as well. Because right now, there's not really a defined method for that. And then I'll pass this over to you, Jean Marc. Yeah. So, yeah, I saw a lot of questions about using different parsers or different type of documents. So this first release is only PDF and CSV, but of course, we want to expand that and there are some open source libraries that allows to pass this type of documents. There are a lot. So the idea is to use this workspace to test different libraries and see which one works the best for you, specific documents and for you, specific data. And so yeah. So plan is, first, I think to support OCR. So if you have documents, PDFs that are scanned, for example. But also also document types, Docix, XLS, PIP, proper point, etcetera, etcetera. And especially semi structured data like JSON, HTML. We have a lot of ideas there. And more parcel, so and secured. Lama pass just released the light pass, which looks good and very quick and come on the different type of format. Open data loader also. So there are quite a lot of new libraries that I I want to test and put here in the server. Also, here, we we wanted to make everything multimodal, so to use tables, images, etcetera. And now we have good multi model embedding, so the idea would be to support that as well and to have more complex, well, ways to pass it as a PDF. Next. On the entity graph, so right now we do one pass extraction. The idea would be to improve because you can do multi pass. This is what we do with customers, so you extract once. And well, first you can divide first extracts entities and then the relationships. And another way is to run the same extraction several time, but to ask the LLM to correct the previous extraction. So you give the chunk, you give what has already been extracted. And you can say check that there is no hallucination or remove the hallucination. Or what should you do to what other nodes and relationship should we extract from that that the first pass missed. What we want also is to be able to update the schema and rerun the extraction only for the diff of the schema. Yeah. Then we need the tool for entity resolution and entity linking and to log and estimate the token usage and the cost of the extraction basically for this particular document, but also at scale. Next. Yeah. Good. Yeah. This is just a slide, on our, startup program. So, if you're interested in our startup program and you wanna check that out, we have a QR code here that you can scan to learn more. Very cool. Thank you, Jean Marc. Thank you, Alex. This was amazing. I I see so many so many so much love in chat as well and and so much for this cool mix of, you know, prepared video, but also live demo and and interesting slides and contents are very, good. Thank you, you know, for this for this amazing workshop today. Two hours went by in a whirlwind, at least for me. I hope it you felt the same. I see, you know, all these questions, all these comments. It's really it's really amazing to see. Thank you very much also, Jennifer, in the in the back and visibly answering questions and and helping out. So I think, yeah, it was a it was a great event today. Thank you for for this. As as I said in the beginning, we recorded this. So if you are you know, missed something, if you wanna watch something again, we will share links to the videos with you or to the one video, obviously, with you afterwards. You will also have a link to all the materials that was shared today by Jean Marc and Alex. You'll see that as well. So you can recover and then rewatch the video from the screencast, but also all the other materials, the slides, the the the other resources. So you should have everything with you in a in a in a moment later today or tomorrow. The links were in chat as well, so you can you can grab that and and and check that out in in in the future. So I hope, yeah, everything should be available. If you have any question, if anything is is still unclear, we have a great way of of exchanging thoughts and ideas on our community forum or on Discord. So hop over there if you are interested in learning more, if you wanna discuss some other things, if you wanna give maybe also feedback. Maybe that's a good question. Right? Alex and Andrew and Mark, what's the the best way for people to to give you feedback if if they have feature requests, anything is maybe broken or something else is yeah. Or also praise, obviously. What's the best channel on this? I think probably GitHub issues are probably a really good way to just raise future requests or bugs that you find with the workspace because those are easily accessible to us, and we'll be able to get those fixed. Super good. I. had maybe if you could quickly stop screen sharing, Alex, because I have only two things I wanted to to get out. Yeah. So here, one thing, if you're interested in in, you know, hearing more about how this how this is is done and what what you can do with with MCP and Aura agents, we have a livestream next week with Luan where we talk about the Nodes AI agenda with with Aura Agents also with with some MCP love in there. So I think that should be interesting as well if you are interested in hearing that. Obviously, Nodes a Nodes AI is coming up. We have one more workshop taking place next week about memory, AI agent memory. So that's that's also happening next week. And, yeah, I I obviously, I forgot again too much. Hopefully, if you're still around, please please let us know how you like this workshop, but I think I I can gather from the the feedback and chat that you liked it. So sorry about that. And, yeah, like I said, let us know in our communities what you what you wanna see. But also Aura Free is available for for deployment in the cloud as a free option, and the Graph Academy is a great place to learn. So if you are now starting with all of this and want to know, hey. I need to hold up. I need to back up a little bit, check out the graph academy. There's a full track on graph rack. This includes all all all kinds of workshops and courses, so you get more hands on workshop there on learning. And, yeah, I think with that, we're at the end, a little overrun. So thank you everybody for sticking around. Thank you for for watching. Obviously, big thank you to, again, to you, Alex and Jean Marc. It was amazing. And I hope to see you soon either next week at the at the final workshop at notes or at an event somewhere else. Take care, everybody. Thank you. Thanks. Thank you.