HippoRAG 2 原理精读

提示：文章写完后，目录可以自动生成，如何生成可参考右边的帮助文档

整体流程

从上图可以看出，整个流程分为两个阶段

1、离线索引阶段

2、在线检索和问答阶段

离线索引阶段

1、对文档进行分段
2、抽取实体

对应的prompt

ner_system = """Your task is to extract named entities from the given paragraph. 
Respond with a JSON list of entities.
"""one_shot_ner_paragraph = """Radio City
Radio City is India's first private FM radio station and was started on 3 July 2001.
It plays Hindi, English and regional songs.
Radio City recently forayed into New Media in May 2008 with the launch of a music portal - PlanetRadiocity.com that offers music related news, videos, songs, and other music-related features."""one_shot_ner_output = """{"named_entities":["Radio City", "India", "3 July 2001", "Hindi", "English", "May 2008", "PlanetRadiocity.com"]
}
"""prompt_template = [{"role": "system", "content": ner_system},{"role": "user", "content": one_shot_ner_paragraph},{"role": "assistant", "content": one_shot_ner_output},{"role": "user", "content": "${passage}"}
]

LLM的输出如

{"named_entities": ["迦楼罗","大鹏金翅鸟","岳飞"]
}

3、根据实体和文本生成三元组

from .ner import one_shot_ner_paragraph, one_shot_ner_output
from ...utils.llm_utils import convert_format_to_templatener_conditioned_re_system = """Your task is to construct an RDF (Resource Description Framework) graph from the given passages and named entity lists. 
Respond with a JSON list of triples, with each triple representing a relationship in the RDF graph. Pay attention to the following requirements:
- Each triple should contain at least one, but preferably two, of the named entities in the list for each passage.
- Clearly resolve pronouns to their specific names to maintain clarity."""ner_conditioned_re_frame = """Convert the paragraph into a JSON dict, it has a named entity list and a triple list.
Paragraph:

{passage}

{named_entity_json}
"""ner_conditioned_re_input = ner_conditioned_re_frame.format(passage=one_shot_ner_paragraph, named_entity_json=one_shot_ner_output)ner_conditioned_re_output = """{"triples": [["Radio City", "located in", "India"],["Radio City", "is", "private FM radio station"],["Radio City", "started on", "3 July 2001"],["Radio City", "plays songs in", "Hindi"],["Radio City", "plays songs in", "English"],["Radio City", "forayed into", "New Media"],["Radio City", "launched", "PlanetRadiocity.com"],["PlanetRadiocity.com", "launched in", "May 2008"],["PlanetRadiocity.com", "is", "music portal"],["PlanetRadiocity.com", "offers", "news"],["PlanetRadiocity.com", "offers", "videos"],["PlanetRadiocity.com", "offers", "songs"]]
}
"""prompt_template = [{"role": "system", "content": ner_conditioned_re_system},{"role": "user", "content": ner_conditioned_re_input},{"role": "assistant", "content": ner_conditioned_re_output},{"role": "user", "content": convert_format_to_template(original_string=ner_conditioned_re_frame, placeholder_mapping=None, static_values=None)}
]

这一步会根据上面抽取出来的实体、原始文本内容生成三元组。

格式如：[主语, 谓语, 宾语]

4、对原始文本、三元组、实体进行向量化

使用的是 nvidia/NV-Embed-v2 模型，参数量为7B

对不同类型的数据，使用的prompt不同，如下：

instructions = {'ner_to_node': 'Given a phrase, retrieve synonymous or relevant phrases that best match this phrase.','query_to_node': 'Given a question, retrieve relevant phrases that are mentioned in this question.','query_to_fact': 'Given a question, retrieve relevant triplet facts that matches this question.','query_to_sentence': 'Given a question, retrieve relevant sentences that best answer the question.','query_to_passage': 'Given a question, retrieve relevant documents that best answer the question.',
}

5、增加同义词边

连接语义相近的实体

在线检索和问答阶段

1、检索topn 的三元组，用LLM对检索结果过滤无关三元组
* 如果没有检索到三元组，则直接检索相关文档
* 如果检索到三元组，则从构建的图中基于个性化的pagerank算法，检索出topn的相关文档

2、利用LLM基于检索结果回答

对应prompt

one_shot_ircot_demo_docs = ("""Wikipedia Title: Milk and Honey (album)\nMilk and Honey is an album by John Lennon and Yoko Ono released in 1984. Following the compilation "The John Lennon Collection", it is Lennon's eighth and final studio album, and the first posthumous release of new Lennon music, having been recorded in the last months of his life during and following the sessions for their 1980 album "Double Fantasy". It was assembled by Yoko Ono in association with the Geffen label.\n\n""""""Wikipedia Title: John Lennon Museum\nJohn Lennon Museum (ジョン・レノン・ミュージアム , Jon Renon Myūjiamu ) was a museum located inside the Saitama Super Arena in Chūō-ku, Saitama, Saitama Prefecture, Japan. It was established to preserve knowledge of John Lennon's life and musical career. It displayed Lennon's widow Yoko Ono's collection of his memorabilia as well as other displays. The museum opened on October 9, 2000, the 60th anniversary of Lennon’s birth, and closed on September 30, 2010, when its exhibit contract with Yoko Ono expired. A tour of the museum began with a welcoming message and short film narrated by Yoko Ono (in Japanese with English headphones available), and ended at an avant-garde styled "reflection room" full of chairs facing a slide show of moving words and images. After this room there was a gift shop with John Lennon memorabilia available.\n\n""""""Wikipedia Title: Walls and Bridges\nWalls and Bridges is the fifth studio album by English musician John Lennon. It was issued by Apple Records on 26 September 1974 in the United States and on 4 October in the United Kingdom. Written, recorded and released during his 18-month separation from Yoko Ono, the album captured Lennon in the midst of his "Lost Weekend". "Walls and Bridges" was an American "Billboard" number-one album and featured two hit singles, "Whatever Gets You thru the Night" and "#9 Dream". The first of these was Lennon's first number-one hit in the United States as a solo artist, and his only chart-topping single in either the US or Britain during his lifetime.\n\n""""""Wikipedia Title: Nobody Loves You (When You're Down and Out)\n"Nobody Loves You (When You're Down and Out)" is a song written by John Lennon released on his 1974 album "Walls and Bridges". The song is included on the 1986 compilation "Menlove Ave.", the 1990 boxset "Lennon", the 1998 boxset "John Lennon Anthology", the 2005 two-disc compilation "", and the 2010 boxset "Gimme Some Truth".\n\n""""""Wikipedia Title: Give Peace a Chance\n"Give Peace a Chance" is an anti-war song written by John Lennon (credited to Lennon–McCartney), and performed with Yoko Ono in Montreal, Quebec, Canada. Released as a single in 1969 by the Plastic Ono Band on Apple Records (catalogue Apple 13 in the United Kingdom, Apple 1809 in the United States), it is the first solo single issued by Lennon, released when he was still a member of the Beatles, and became an anthem of the American anti-war movement during the 1970s. It peaked at number 14 on the "Billboard" Hot 100 and number 2 on the British singles chart.\n"""
)one_shot_ircot_demo = (f'{one_shot_ircot_demo_docs}''\n\nQuestion: 'f"Nobody Loves You was written by John Lennon and released on what album that was issued by Apple Records, and was written, recorded, and released during his 18 month separation from Yoko Ono?"'\nThought: 'f"The album issued by Apple Records, and written, recorded, and released during John Lennon's 18 month separation from Yoko Ono is Walls and Bridges. Nobody Loves You was written by John Lennon on Walls and Bridges album. So the answer is: Walls and Bridges."'\n\n'
)ircot_system = ('You serve as an intelligent assistant, adept at facilitating users through complex, multi-hop reasoning across multiple documents. This task is illustrated through demonstrations, each consisting of a document set paired with a relevant question and its multi-hop reasoning thoughts. Your task is to generate one thought for current step, DON\'T generate the whole thoughts at once! If you reach what you believe to be the final step, start with "So the answer is:".''\n\n'f'{one_shot_ircot_demo}'
)prompt_template = [{"role": "system", "content": ircot_system},{"role": "user", "content": "${prompt_user}"}
]

总结

1、只是用三元组来协助检索，并没有利用图
2、难以相信这种做法能超越普通RAG几十个点

参考：

HippoRAG 2: From RAGtoMemory: Non-Parametric Continual Learning for Large Language Models