Dependencies:

Install coreferee dependencies with spacy and other related packages like numpy, etc
pip install llama-index-graph-stores-neo4j-check for llamaindex and llamaindex core packages
pip install ne04j
download coreferee model
download spacy model en_core_web_lg
Install neo4j graph database application


Alternative for non usage of llm in retreival part:

1.We can make use of entities from query and extract the required ones
2.Need to use the extracted entity from query and construct a cypher query with entity to find the node
3.After confirming the node with entity deduplication, construct a cypher query for relationships and find the other nodes
4.Use the other nodes for KG retrieval response generation


INTEGRATION STEPS:
1.Setup the neo4j in Linux for dev,demo and qa instances.(backup identify)
2.Define the ingestion pipeline for data sources and construct a class for KG ingestion in current framework
3.Modify the class so that it can be used for long term memory retrieval for agent 
4.Also prepare the class to allow for any product recommendations from KG nodes to user based queries.
5.Setup the retrieval pipeline to extract the relevant nodes and generate the responses
6.Allow the setup to use KG so that when no contexts from RAG are outputted, use the KG based fallback and also, allow for 
alternative use of KG in current framework.










API integration:

1.Need to use the python api as a background task during the data sources ingestion for new data and for old data need to develope task for the old data

2.Need to setup backup data for neo4j same as performed in opensearch whenever server gets restarted or faulted.




Python integration:
1.Setup the packages required for new integration of KG in current framework
2.Prepare a Pipeline Class  for KG 
3.prepare a api for ingestion pipeline which will act as a background task
4.integrate the retrieval in the current agentic RAG integration with fallback as opensearch
5.Deduplication of entities include in pipeline and Entity Normalization
6.Architectural issues fixes: batching, parallel processing, metadata, fault tolerance and observability
7.Add Hybrid Retrieval and improve prompt retrieval(Graph Quality Filters) into data retrieval  
8.Neo4j Indexing
9.Hybrid Graph RAG combine vector retrieval and graph traversal


UI integration
1.To give any sample graph on the query asked by the user , the nodes and relationships.(To be discussed)