{"id":19357,"date":"2025-01-25T21:46:54","date_gmt":"2025-01-25T13:46:54","guid":{"rendered":"https:\/\/www.aisharenet.com\/?p=19357"},"modified":"2025-07-14T09:40:45","modified_gmt":"2025-07-14T01:40:45","slug":"ragas","status":"publish","type":"post","link":"https:\/\/www.kdjingpai.com\/pt\/ragas\/","title":{"rendered":"Ragas\uff1a\u8bc4\u4f30RAG\u53ec\u56deQA\u51c6\u786e\u7387\u4e0e\u7b54\u6848\u76f8\u5173\u6027"},"content":{"rendered":"<p>Ragas\u662f\u4e00\u4e2a\u4e13\u95e8\u7528\u4e8e\u8bc4\u4f30\u548c\u4f18\u5316\u68c0\u7d22\u589e\u5f3a\u751f\u6210\uff08RAG\uff09\u7cfb\u7edf\u7684\u5de5\u5177\u3002\u5b83\u901a\u8fc7\u5206\u6790\u67e5\u8be2\u3001\u68c0\u7d22\u4e0a\u4e0b\u6587\u548c\u751f\u6210\u7b54\u6848\u4e4b\u95f4\u7684\u5173\u7cfb\uff0c\u63d0\u4f9b\u4e86\u4e00\u5957\u5168\u9762\u7684\u8bc4\u4f30\u6307\u6807\u3002\u8fd9\u4e9b\u6307\u6807\u5305\u62ec\u5fe0\u5b9e\u5ea6\u3001\u7b54\u6848\u76f8\u5173\u6027\u3001\u4e0a\u4e0b\u6587\u76f8\u5173\u6027\u3001\u4e0a\u4e0b\u6587\u53ec\u56de\u7387\u548c\u4e0a\u4e0b\u6587\u7cbe\u786e\u5ea6\u3002Ragas\u5e2e\u52a9\u5f00\u53d1\u8005\u8bc4\u4f30\u548c\u4f18\u5316RAG\u7cfb\u7edf\u7684\u6027\u80fd\uff0c\u786e\u4fdd\u751f\u6210\u7684\u7b54\u6848\u51c6\u786e\u3001\u76f8\u5173\u4e14\u57fa\u4e8e\u68c0\u7d22\u5185\u5bb9\u3002\u65e0\u8bba\u662f\u5f00\u53d1\u65b0\u7cfb\u7edf\u8fd8\u662f\u4f18\u5316\u73b0\u6709\u7cfb\u7edf\uff0cRagas\u90fd\u80fd\u63d0\u4f9b\u5f3a\u5927\u7684\u652f\u6301\u3002<\/p>\n<p>&nbsp;<\/p>\n<h2>\u529f\u80fd\u5217\u8868<\/h2>\n<ul>\n<li><strong>\u5fe0\u5b9e\u5ea6\u8bc4\u4f30<\/strong>\uff1a\u8861\u91cf\u751f\u6210\u7684\u7b54\u6848\u5bf9\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u7684\u5fe0\u5b9e\u7a0b\u5ea6\u3002<\/li>\n<li><strong>\u7b54\u6848\u76f8\u5173\u6027\u8bc4\u4f30<\/strong>\uff1a\u8bc4\u4f30\u751f\u6210\u7684\u7b54\u6848\u4e0e\u539f\u59cb\u67e5\u8be2\u7684\u76f8\u5173\u7a0b\u5ea6\u3002<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\u76f8\u5173\u6027\u8bc4\u4f30<\/strong>\uff1a\u8861\u91cf\u68c0\u7d22\u51fa\u6765\u7684\u4fe1\u606f\u6709\u591a\u8d34\u5408\u95ee\u9898\u3002<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\u53ec\u56de\u7387\u8bc4\u4f30<\/strong>\uff1a\u8bc4\u4f30\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u662f\u5426\u5305\u542b\u56de\u7b54\u95ee\u9898\u6240\u9700\u7684\u5168\u90e8\u4fe1\u606f\u3002<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\u7cbe\u786e\u5ea6\u8bc4\u4f30<\/strong>\uff1a\u8861\u91cf\u68c0\u7d22\u5230\u7684\u76f8\u5173\u4e0a\u4e0b\u6587\u5728\u7ed3\u679c\u4e2d\u7684\u6392\u5e8f\u8d28\u91cf\u3002<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>\u4f7f\u7528\u5e2e\u52a9<\/h2>\n<h3>\u5b89\u88c5\u6d41\u7a0b<\/h3>\n<ol>\n<li><strong>\u901a\u8fc7PyPI\u5b89\u88c5<\/strong>\uff1a<\/li>\n<\/ol>\n<pre><code>   pip install ragas\r\n<\/code><\/pre>\n<ol start=\"2\">\n<li><strong>\u4ece\u6e90\u7801\u5b89\u88c5<\/strong>\uff1a<\/li>\n<\/ol>\n<pre><code>   pip install git+https:\/\/github.com\/explodinggradients\/ragas\r\n<\/code><\/pre>\n<h3>\u5feb\u901f\u5f00\u59cb<\/h3>\n<ol>\n<li><strong>\u5bfc\u5165\u5fc5\u8981\u6a21\u5757<\/strong>\uff1a<\/li>\n<\/ol>\n<pre><code>   from ragas import SingleTurnSample\r\nfrom ragas.metrics import AspectCritic\r\n<\/code><\/pre>\n<ol start=\"2\">\n<li><strong>\u51c6\u5907\u6d4b\u8bd5\u6570\u636e<\/strong>\uff1a<\/li>\n<\/ol>\n<pre><code>   test_data = {\r\n\"user_input\": \"summarise given text\\nThe company reported an 8% rise in Q3 2024, driven by strong performance in the Asian market. Sales in this region have significantly contributed to the overall growth. Analysts attribute this success to strategic marketing and product localization. The positive trend in the Asian market is expected to <a href=\"https:\/\/www.kdjingpai.com\/de\/continue\/\">continue<\/a> into the next quarter.\",\r\n\"response\": \"The company experienced an 8% increase in Q3 2024, largely due to effective marketing strategies and product adaptation, with expectations of continued growth in the coming quarter.\"\r\n}\r\n<\/code><\/pre>\n<ol start=\"3\">\n<li><strong>\u8bc4\u4f30RAG\u7cfb\u7edf<\/strong>\uff1a<\/li>\n<\/ol>\n<pre><code>   evaluator_llm = LangchainLLMWrapper(ChatOpenAI(model=\"gpt-4o\"))\r\nmetric = AspectCritic(name=\"summary_accuracy\", llm=evaluator_llm, definition=\"Verify if the summary is accurate.\")\r\nawait metric.single_turn_ascore(SingleTurnSample(**test_data))\r\n<\/code><\/pre>\n<h3>\u8be6\u7ec6\u529f\u80fd\u64cd\u4f5c\u6d41\u7a0b<\/h3>\n<ol>\n<li><strong>\u5fe0\u5b9e\u5ea6\u8bc4\u4f30<\/strong>\uff1a\n<ul>\n<li>\u4f7f\u7528Ragas\u63d0\u4f9b\u7684\u5fe0\u5b9e\u5ea6\u6307\u6807\uff0c\u8bc4\u4f30\u751f\u6210\u7684\u7b54\u6848\u662f\u5426\u5fe0\u5b9e\u4e8e\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u3002<\/li>\n<li>\u901a\u8fc7\u8c03\u7528\u76f8\u5e94\u7684\u8bc4\u4f30\u51fd\u6570\uff0c\u83b7\u53d6\u5fe0\u5b9e\u5ea6\u8bc4\u4f30\u7ed3\u679c\u5e76\u8fdb\u884c\u5206\u6790\u3002<\/li>\n<\/ul>\n<\/li>\n<li><strong>\u7b54\u6848\u76f8\u5173\u6027\u8bc4\u4f30<\/strong>\uff1a\n<ul>\n<li>\u4f7f\u7528Ragas\u7684\u7b54\u6848\u76f8\u5173\u6027\u6307\u6807\uff0c\u8bc4\u4f30\u751f\u6210\u7684\u7b54\u6848\u4e0e\u539f\u59cb\u67e5\u8be2\u7684\u76f8\u5173\u7a0b\u5ea6\u3002<\/li>\n<li>\u901a\u8fc7\u914d\u7f6e\u76f8\u5173\u6027\u8bc4\u4f30\u53c2\u6570\uff0c\u5b9a\u5236\u5316\u8bc4\u4f30\u751f\u6210\u7b54\u6848\u7684\u76f8\u5173\u6027\u3002<\/li>\n<\/ul>\n<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\u76f8\u5173\u6027\u8bc4\u4f30<\/strong>\uff1a\n<ul>\n<li>\u4f7f\u7528Ragas\u7684\u4e0a\u4e0b\u6587\u76f8\u5173\u6027\u6307\u6807\uff0c\u8861\u91cf\u68c0\u7d22\u51fa\u6765\u7684\u4fe1\u606f\u6709\u591a\u8d34\u5408\u95ee\u9898\u3002<\/li>\n<li>\u901a\u8fc7\u8c03\u7528\u76f8\u5173\u6027\u8bc4\u4f30\u51fd\u6570\uff0c\u83b7\u53d6\u4e0a\u4e0b\u6587\u76f8\u5173\u6027\u8bc4\u4f30\u7ed3\u679c\u3002<\/li>\n<\/ul>\n<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\u53ec\u56de\u7387\u8bc4\u4f30<\/strong>\uff1a\n<ul>\n<li>\u4f7f\u7528Ragas\u7684\u4e0a\u4e0b\u6587\u53ec\u56de\u7387\u6307\u6807\uff0c\u8bc4\u4f30\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u662f\u5426\u5305\u542b\u56de\u7b54\u95ee\u9898\u6240\u9700\u7684\u5168\u90e8\u4fe1\u606f\u3002<\/li>\n<li>\u901a\u8fc7\u8bbe\u7f6e\u53ec\u56de\u7387\u8bc4\u4f30\u53c2\u6570\uff0c\u786e\u4fdd\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u5168\u9762\u8986\u76d6\u95ee\u9898\u6240\u9700\u7684\u4fe1\u606f\u3002<\/li>\n<\/ul>\n<\/li>\n<li><strong>\u4e0a\u4e0b\u6587\u7cbe\u786e\u5ea6\u8bc4\u4f30<\/strong>\uff1a\n<ul>\n<li>\u4f7f\u7528Ragas\u7684\u4e0a\u4e0b\u6587\u7cbe\u786e\u5ea6\u6307\u6807\uff0c\u8861\u91cf\u68c0\u7d22\u5230\u7684\u76f8\u5173\u4e0a\u4e0b\u6587\u5728\u7ed3\u679c\u4e2d\u7684\u6392\u5e8f\u8d28\u91cf\u3002<\/li>\n<li>\u901a\u8fc7\u914d\u7f6e\u7cbe\u786e\u5ea6\u8bc4\u4f30\u53c2\u6570\uff0c\u4f18\u5316\u68c0\u7d22\u7ed3\u679c\u7684\u6392\u5e8f\u3002<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<h2>Ragas\u8bc4\u4f30QA\u7b14\u8bb0<\/h2>\n<p>QA\u53ec\u56de\u4e00\u822c\u5305\u542b\u4e24\u4e2a\u7ec4\u4ef6\uff1a<br \/>\n1\u3001\u68c0\u7d22\u5668 &#8211; \u68c0\u7d22\u56de\u7b54\u67e5\u8be2\u6240\u9700\u7684\u6700\u76f8\u5173\u4fe1\u606f\u3002<br \/>\n2\u3001\u751f\u6210\u5668 &#8211; \u4f7f\u7528\u68c0\u7d22\u5230\u7684\u4fe1\u606f\u751f\u6210\u7b54\u6848\u3002<\/p>\n<p>&nbsp;<\/p>\n<p>\u8bc4\u4f30QA\u51c6\u786e\u7387\u4e00\u76f4\u662f\u4e2a\u5927\u95ee\u9898\u3002<a href=\"https:\/\/aclanthology.org\/W04-1013\/?ref=blog.langchain.dev\" target=\"_blank\" rel=\"noopener\" aria-label=\"https:\/\/aclanthology.org\/W04-1013\/?ref=blog.langchain.dev\">ROUGE<\/a>\uff08<a href=\"https:\/\/aclanthology.org\/W04-1013.pdf\" target=\"_blank\" rel=\"noopener\" aria-label=\"https:\/\/aclanthology.org\/W04-1013.pdf\">A Package for Automatic Evaluation of Summaries<\/a>\uff09\u548c<a href=\"https:\/\/dl.acm.org\/doi\/10.3115\/1073083.1073135?ref=blog.langchain.dev\" target=\"_blank\" rel=\"noopener\" aria-label=\"https:\/\/dl.acm.org\/doi\/10.3115\/1073083.1073135?ref=blog.langchain.dev\">BLUE<\/a>\uff08\u00a0a method for automatic evaluation of machine translation\uff09\u6548\u679c\u6bd4\u8f83\u5dee\u3002<\/p>\n<p>\u76ee\u524d\u65b9\u6cd5\uff1a\u5229\u7528\u5f3a\u5927\u7684LLM\u8fdb\u884c\u65e0\u53c2\u8003\u8bc4\u4f30\uff0c\u6240\u4ee5\u8981\u4ed4\u7ec6\u804a\u804aRagas\u3002<\/p>\n<p>Ragas \u65e8\u5728\u89e3\u51b3\u4f7f\u7528 LLM \u8bc4\u4f30\u60a8\u7684 QA \u7ba1\u9053\u7684\u8fd9\u4e9b\u9650\u5236\uff0c\u540c\u65f6\u8fd8\u4f7f\u7528\u5c3d\u53ef\u80fd\u5c11\u3001\u66f4\u7ecf\u6d4e\u3001\u66f4\u5feb\u6377\u7684\u6ce8\u91ca\u6570\u636e\u63d0\u4f9b\u53ef\u64cd\u4f5c\u7684\u6307\u6807\u3002<\/p>\n<p>Ragas \u662f\u4e00\u4e2a\u6846\u67b6\uff0c\u53ef\u5e2e\u52a9\u60a8\u8bc4\u4f30\u8fd9\u4e9b\u4e0d\u540c\u65b9\u9762\u7684 QA \u7ba1\u9053\u3002\u5b83\u4e3a\u60a8\u63d0\u4f9b\u4e86\u4e00\u4e9b\u6307\u6807\u6765\u8bc4\u4f30QA\u7cfb\u7edf\u7684\u4e0d\u540c\u65b9\u9762\u3002<\/p>\n<p><img decoding=\"async\" title=\"Ragas\uff1a\u8bc4\u4f30RAG\u53ec\u56de\u51c6\u786e\u7387\u4e0e\u7b54\u6848\u76f8\u5173\u6027-1\" src=\"https:\/\/www.kdjingpai.com\/wp-content\/uploads\/2025\/01\/86ff5bb5e101b90.jpg\" alt=\"Ragas\uff1a\u8bc4\u4f30RAG\u53ec\u56de\u51c6\u786e\u7387\u4e0e\u7b54\u6848\u76f8\u5173\u6027-1\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><strong>\u7528\u4e8e\u8bc4\u4f30\u68c0\u7d22\u7684\u6307\u6807\uff1a<\/strong>\u00a0\u63d0\u4f9b\uff0c\u5e76\u4e3a\u60a8\u63d0\u4f9b\u68c0\u7d22\u7cfb\u7edf\u6027\u80fd\u7684\u8861\u91cf\u6807\u51c6\u3002<br \/>\ncontext_relevancy<br \/>\ncontext_recall<\/p>\n<p><strong>\u8bc4\u4f30\u751f\u6210\u7684\u6307\u6807\uff1a<\/strong>\u00a0\u63d0\u4f9b\u54ea\u4e9b\u8861\u91cf\u5e7b\u89c9\uff0c\u54ea\u4e9b\u8861\u91cf\u5982\u4f55\u56de\u7b54\u95ee\u9898\u3002<br \/>\nfaithfulness<br \/>\nanswer_relevancy<\/p>\n<h3>faithfulness<\/h3>\n<p>\u6839\u636e\u63d0\u4f9b\u7684\u4e0a\u4e0b\u6587\u8861\u91cf\u751f\u6210\u7684\u7b54\u6848\u7684\u4e8b\u5b9e\u51c6\u786e\u6027\u3002<\/p>\n<p>\u8fd9\u5206\u4e24\u6b65\u5b8c\u6210\u3002<\/p>\n<p>\u9996\u5148\uff0c\u7ed9\u5b9a\u4e00\u4e2a\u95ee\u9898\u5e76\u751f\u6210\u7b54\u6848\uff0cRagas \u4f7f\u7528 LLM \u6765\u627e\u51fa\u751f\u6210\u7684\u7b54\u6848\u6240\u505a\u7684\u9648\u8ff0\u3002\u8fd9\u7ed9\u51fa\u4e86\u4e00\u4e2a\u8bed\u53e5\u5217\u8868\uff0c\u6211\u4eec\u5fc5\u987b\u68c0\u67e5\u5176\u6709\u6548\u6027\u3002\u5728\u6b65\u9aa4 2 \u4e2d\uff0c\u7ed9\u5b9a\u8bed\u53e5\u5217\u8868\u548c\u8fd4\u56de\u7684\u4e0a\u4e0b\u6587\uff0cRagas \u4f7f\u7528 LLM \u68c0\u67e5\u63d0\u4f9b\u7684\u8bed\u53e5\u662f\u5426\u53d7\u4e0a\u4e0b\u6587\u652f\u6301\u3002\u5c06\u6b63\u786e\u9648\u8ff0\u7684\u6570\u91cf\u76f8\u52a0\u5e76\u9664\u4ee5\u751f\u6210\u7684\u7b54\u6848\u4e2d\u7684\u9648\u8ff0\u603b\u6570\uff0c\u4ee5\u83b7\u5f97\u7ed9\u5b9a\u793a\u4f8b\u7684\u5206\u6570\u3002<\/p>\n<p><strong>\u6b65\u9aa4\u4e00\uff1a<\/strong><\/p>\n<p>\u6839\u636e\u7b2c\u4e00\u4e2aprompt:<\/p>\n<pre><code>Given a question and answer, create one or more statements from answer.\r\nquestion: Who was Albert Einstein and what is he best known for?\r\nanswer: He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics.\r\nstatements:\r\nAlbert Einstein was born in Germany.\r\nAlbert Einstein was best known for his theory of relativity.\r\nquestion: Cadmium Chloride is slightly soluble in this chemical, it is also called what?\r\nanswer: alochol\r\nstatements:\r\nCadmium Chloride is slightly soluble in alcohol.\r\nquestion: Were Shahul and Jithin of the same nationality?\r\nanswer: They were from different countries.\r\nstatements:\r\nShahul and Jithin were from different countries.\r\nquestion:\u81b3\u98df\u5206\u6790\u662f\u4ec0\u4e48?\r\nanswer: \u81b3\u98df\u5206\u6790\u662f\u4e00\u79cd\u8bc4\u4f30\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u7684\u65b9\u6cd5\uff0c\u901a\u8fc7\u8be6\u7ec6\u5206\u6790\u4e00\u5468\u5185\u6240\u5403\u7684\u98df\u7269\uff0c\u6765\u786e\u5b9a\u4eba\u4eec\u6444\u5165\u7684\u8425\u517b\u7d20\u79cd\u7c7b\u548c\u6570\u91cf\u3002\u5b83\u53ef\u4ee5\u63ed\u793a\u4eba\u4eec\u662f\u5426\u7f3a\u4e4f\u67d0\u4e9b\u8425\u517b\u7d20\uff0c\u4ee5\u53ca\u98df\u7269\u4e2d\u662f\u5426\u5b58\u5728\u4e00\u4e9b\u5bf9\u5065\u5eb7\u6709\u5bb3\u7684\u7269\u8d28\uff0c\u5982\u7cd6\u3001\u76d0\u3001\u5496\u5561\u56e0\u3001\u9152\u7cbe\u3001\u98df\u54c1\u6dfb\u52a0\u5242\u548c\u9632\u8150\u5242\u3002\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\u7684\u8bc4\u4f30\u3002\u7136\u800c\uff0c\u81b3\u98df\u5206\u6790\u5e76\u4e0d\u80fd\u5168\u9762\u8003\u8651\u5230\u98df\u7269\u4e2d\u8425\u517b\u7d20\u542b\u91cf\u7684\u53d8\u5f02\u6027\u3001\u4e2a\u4eba\u7684\u8425\u517b\u7d20\u9700\u6c42\uff0c\u4ee5\u53ca\u8425\u517b\u7d20\u7684\u5438\u6536\u548c\u5229\u7528\u60c5\u51b5\u3002\u56e0\u6b64\uff0c\u5b83\u53ef\u80fd\u5e76\u4e0d\u662f\u8bc4\u4f30\u4e2a\u4eba\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u6700\u51c6\u786e\u7684\u65b9\u6cd5\u3002\r\nstatements:\r\n<\/code><\/pre>\n<p>\u5f97\u5230\u7ed3\u679c1\uff1a<\/p>\n<pre><code>['\u81b3\u98df\u5206\u6790\u662f\u4e00\u79cd\u8bc4\u4f30\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u7684\u65b9\u6cd5\u3002', \r\n'\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u786e\u5b9a\u4eba\u4eec\u6444\u5165\u7684\u8425\u517b\u7d20\u79cd\u7c7b\u548c\u6570\u91cf\u3002', \r\n'\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63ed\u793a\u4eba\u4eec\u662f\u5426\u7f3a\u4e4f\u67d0\u4e9b\u8425\u517b\u7d20\u3002', \r\n'\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63ed\u793a\u98df\u7269\u4e2d\u662f\u5426\u5b58\u5728\u4e00\u4e9b\u5bf9\u5065\u5eb7\u6709\u5bb3\u7684\u7269\u8d28\u3002', \r\n'\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\u7684\u8bc4\u4f30\u3002', \r\n'\u81b3\u98df\u5206\u6790\u53ef\u80fd\u5e76\u4e0d\u662f\u8bc4\u4f30\u4e2a\u4eba\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u6700\u51c6\u786e\u7684\u65b9\u6cd5\u3002']\r\n<\/code><\/pre>\n<p><strong>\u6b65\u9aa4\u4e8c\uff1a<\/strong><\/p>\n<p>\u6839\u636e\u7b2c\u4e8c\u4e2aprompt:<\/p>\n<pre><code>Prompt: Natural language inference\r\nConsider the given context and following statements, then determine whether they are supported by the information present in the context.Provide a brief explanation for each statement before arriving at the verdict (Yes\/No). Provide a final verdict for each statement in order at the end in the given format. Do not deviate from the specified format.\r\nContext:\r\nJohn is a student at XYZ University. He is pursuing a degree in <a href=\"https:\/\/www.kdjingpai.com\/de\/tldraw-computer\/\">Computer<\/a> Science. He is enrolled in several courses this semester, including Data Structures, Algorithms, and Database Management. John is a diligent student and spends a significant amount of time studying and completing assignments. He often stays late in the library to work on his projects.\r\nstatements:\r\n1. John is majoring in Biology.\r\n2. John is taking a course on Artificial Intelligence.\r\n3. John is a dedicated student.\r\n4. John has a part-time job.\r\n5. John is interested in computer programming.\r\nAnswer:\r\n1. John is majoring in Biology.\r\nExplanation: John's major is explicitly mentioned as Computer Science. There is no information suggesting he is majoring in Biology.  Verdict: No.\r\n2. John is taking a course on Artificial Intelligence.\r\nExplanation: The context mentions the courses John is currently enrolled in, and Artificial Intelligence is not mentioned. Therefore, it cannot be deduced that John is taking a course on AI. Verdict: No.\r\n3. John is a dedicated student.\r\nExplanation: The prompt states that he spends a significant amount of time studying and completing assignments. Additionally, it mentions that he often stays late in the library to work on his projects, which implies dedication. Verdict: Yes.\r\n4. John has a part-time job.\r\nExplanation: There is no information given in the context about John having a part-time job. Therefore, it cannot be deduced that John has a part-time job.  Verdict: No.\r\n5. John is interested in computer programming.\r\nExplanation: The context states that John is pursuing a degree in Computer Science, which implies an interest in computer programming. Verdict: Yes.\r\nFinal verdict for each statement in order: No. No. Yes. No. Yes.\r\ncontext:\r\n\u4f60\u60f3\u8981\u8fbe\u5230\u4ec0\u4e48\u6837\u7684\u5065\u5eb7\u7a0b\u5ea6\uff1f\u5982\u679c\u4f60\u60f3\u4e86\u89e3\u81ea\u5df1\u7cbe\u795e\u548c\u8eab\u4f53\u7684\u5168\u90e8\u6f5c\u5728\u80fd\u529b\uff0c\u786e\u5b9a\u4f60\u7684\u6700\u4f73\u8425\u517b\u9700\u6c42\u662f\u975e\u5e38\u5fc5\u8981\u7684\u3002\u4f46\u662f\u5982\u679c\u4f60\u7684\u8425\u517b\u9700\u6c42\u5f88\u7279\u522b\uff0c\u90a3\u4e48\u4f60\u8be5\u5982\u4f55\u53d1\u73b0\u8fd9\u4e00\u70b9\u5462\uff1f\u4ece1980\u5e74\u5f00\u59cb\uff0c\u6211\u5236\u5b9a\u5e76\u5b8c\u5584\u4e86\u4e00\u5957\u5206\u6790\u4eba\u4f53\u8425\u517b\u9700\u6c42\u7684\u4f53\u7cfb\uff0c\u6b64\u4f53\u7cfb\u5efa\u7acb\u5728\u8bc4\u4f30\u5f71\u54cd\u4e2a\u4eba\u8425\u517b\u9700\u6c42\u7684\u4e3b\u8981\u56e0\u7d20\u7684\u57fa\u7840\u4e0a\u3002\u6b64\u4f53\u7cfb\u5df2\u7ecf\u572810\u4e07\u4eba\u5f53\u4e2d\u8fdb\u884c\u4e86\u6d4b\u8bd5\u5e76\u5f97\u5230\u4e86\u9a8c\u8bc1\uff0c\u73b0\u5728\u5df2\u88ab\u4e16\u754c\u5404\u56fd\u7684\u4e34\u5e8a\u8425\u517b\u5b66\u5bb6\u6240\u5e94\u7528\u3002\u65e0\u6570\u4eba\u5df2\u7ecf\u4ece\u4e2d\u83b7\u76ca\uff0c\u56e0\u6b64\uff0c\u6211\u77e5\u9053\u4ece\u8fd9\u4e2a\u4f53\u7cfb\u4e2d\u53ef\u4ee5\u5f97\u5230\u4ec0\u4e48\u6837\u7684\u7ed3\u679c\u3002\u8fd9\u4e9b\u7ed3\u679c\u5305\u62ec\uff1a\u66f4\u654f\u9510\u7684\u601d\u7ef4\u3001\u66f4\u597d\u7684\u8bb0\u5fc6\u529b\u3001\u66f4\u5f3a\u7684\u4f53\u529b\u3001\u66f4\u597d\u7684\u4f53\u91cd\u63a7\u5236\u6548\u679c\u3001\u964d\u4f4e\u80c6\u56fa\u9187\u542b\u91cf\u3001\u4f7f\u5f97\u75c5\u60c5\u597d\u8f6c\u3002\u5c3d\u7ba1\u591a\u6570\u88ab\u8bca\u65ad\u4e3a\u6709\u75c5\u7684\u4eba\u5df2\u7ecf\u53d7\u76ca\u4e8e\u4e2a\u4eba\u5065\u5eb7\u8ba1\u5212\uff0c\u4f46\u8fd9\u4e2a\u5065\u5eb7\u8ba1\u5212\u662f\u4ee5\u9884\u9632\u4e3a\u4e3b\u8981\u76ee\u6807\uff0c\u800c\u4e0d\u662f\u4ee5\u6cbb\u75c5\u4e3a\u4e3b\u7684\u3002\u5982\u679c\u4f60\u6b63\u5728\u8fdb\u884c\u533b\u836f\u6cbb\u7597\uff0c\u8bf7\u786e\u8ba4\u8fd9\u4e2a\u8425\u517b\u8865\u5145\u65b9\u6848\u4e0e\u4f60\u63a5\u53d7\u7684\u6cbb\u7597\u63aa\u65bd\u662f\u5426\u80fd\u591f\u5f88\u597d\u5730\u76f8\u5bb9\u3002\u5f71\u54cd\u4f60\u8425\u517b\u9700\u6c42\u7684\u56e0\u7d20\uff0c\u81f3\u5c11\u67098\u9879\u56e0\u7d20\u4f1a\u5f71\u54cd\u4f60\u7684\u8425\u517b\u9700\u6c42\u91cf\u3002\u5e74\u9f84\u3001\u6027\u522b\u4ee5\u53ca\u8fd0\u52a8\u91cf\u90fd\u662f\u5f88\u5e38\u89c1\u7684\u5f71\u54cd\u56e0\u7d20\uff0c\u4f46\u6c61\u67d3\u3001\u538b\u529b\u3001\u57fa\u56e0\u9057\u4f20\u3001\u4f60\u8fc7\u53bb\u7684\u5065\u5eb7\u72b6\u51b5\uff0c\u5f53\u7136\u8fd8\u6709\u81b3\u98df\u6240\u80fd\u63d0\u4f9b\u7684\u8425\u517b\u7d20\u4ee5\u53ca\u53cd\u8425\u517b\u7269\u8d28\uff0c\u5374\u4e0d\u662f\u80fd\u591f\u8f7b\u6613\u4e86\u89e3\u7684\u3002\u4f46\u662f\u6240\u6709\u7684\u8fd9\u4e9b\u7ec6\u8282\u4ee5\u53ca\u66f4\u591a\u7684\u56e0\u7d20\uff0c\u90fd\u5fc5\u987b\u7eb3\u5165\u8003\u8651\u5f53\u4e2d\u3002\u901a\u8fc74\u79cd\u5206\u6790\u65b9\u6cd5\u2014\u2014\u81b3\u98df\u5206\u6790\u3001\u751f\u5316\u5206\u6790\u3001\u75c7\u72b6\u5206\u6790\u3001\u751f\u6d3b\u65b9\u5f0f\u5206\u6790\uff0c\u53ef\u4ee5\u4e86\u89e3\u8fd9\u4e9b\u7ec6\u8282\u3002\u81b3\u98df\u5206\u6790\u8fd9\u79cd\u65b9\u5f0f\u4f3c\u4e4e\u662f\u4ece\u5bb9\u6613\u7740\u624b\u7684\u5730\u65b9\u5f00\u59cb\uff1a\u641e\u6e05\u695a\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u662f\u4ec0\u4e48\uff0c\u5c31\u80fd\u53d1\u73b0\u7f3a\u4e4f\u7684\u8425\u517b\u7d20\u662f\u4ec0\u4e48\u3002\r\n\u4f46\u4e0d\u5e78\u7684\u662f\uff0c\u5982\u679c\u5bf9\u4e00\u5468\u5185\u6240\u5403\u7684\u98df\u7269\u8fdb\u884c\u8be6\u7ec6\u5206\u6790\uff0c\u5e76\u4e0d\u80fd\u7efc\u5408\u8003\u8651\u5230\u98df\u7269\u8425\u517b\u7d20\u542b\u91cf\u7684\u53d8\u5f02\u6027\u3001\u4e2a\u4eba\u7684\u8425\u517b\u7d20\u9700\u6c42\uff0c\u6216\u8005\u8425\u517b\u7d20\u662f\u5426\u80fd\u88ab\u5438\u6536\u3001\u5438\u6536\u540e\u7684\u8425\u517b\u7d20\u53c8\u662f\u5426\u80fd\u88ab\u6709\u6548\u5730\u5229\u7528\u7b49\u60c5\u51b5\u3002\u6211\u89c1\u8fc7\u8bb8\u591a\u4eba\u7684\u996e\u98df\u8d28\u91cf\u5f88\u9ad8\uff0c\u5374\u4f9d\u7136\u6709\u7ef4\u751f\u7d20\u7f3a\u4e4f\u7684\u75c7\u72b6\u3002\u5bf9\u5176\u4e2d\u7edd\u5927\u90e8\u5206\u4eba\u6765\u8bf4\uff0c\u95ee\u9898\u5728\u4e8e\u8425\u517b\u7d20\u5438\u6536\u4e0d\u826f\u3002\u8fd9\u4e9b\u53ef\u53d8\u56e0\u7d20\uff0c\u4f7f\u5f97\u67d0\u4e9b\u901a\u8fc7\u8ba1\u7b97\u673a\u8fdb\u884c\u7684\u81b3\u98df\u5206\u6790\u4e0d\u5982\u8bbe\u60f3\u7684\u51c6\u786e\u3002\u81b3\u98df\u5206\u6790\u5bf9\u4e8e\u8bc4\u4f30\u90a3\u4e9b\u80fd\u5f71\u54cd\u5230\u4eba\u4eec\u8425\u517b\u7d20\u9700\u6c42\u7684\u98df\u7269\u5f88\u6709\u7528\u5904\uff0c\u6bd4\u5982\u7cd6\u3001\u76d0\u3001\u5496\u5561\u3001\u8336\u3001\u9152\u7cbe\u3001\u98df\u54c1\u6dfb\u52a0\u5242\u548c\u9632\u8150\u5242\u3002\u53e6\u4e00\u4e9b\u56e0\u7d20\uff0c\u5982\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\uff0c\u4e5f\u53ef\u7531\u81b3\u98df\u5206\u6790\u8fdb\u884c\u8bc4\u4f30\u3002\u8bf8\u5982\u5934\u53d1\u77ff\u7269\u8d28\u5206\u6790\u4e0e\u7ef4\u751f\u7d20\u8840\u6db2\u5206\u6790\u8fd9\u6837\u7684\u5b9e\u9a8c,\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u4f53\u5185\u751f\u7269\u5316\u5b66\u72b6\u6001\u7684\u786e\u5207\u4fe1\u606f\uff0c\u4e5f\u80fd\u8ba9\u8425\u517b\u987e\u95ee\u4e86\u89e3\u4f60\u8eab\u4f53\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u3002\u4f46\u5e76\u4e0d\u662f\u6240\u6709\u8fd9\u4e9b\u5b9e\u9a8c\u90fd\u80fd\u5bf9\u89c4\u5212\u4f60\u7684\u8425\u517b\u65b9\u6848\u63d0\u4f9b\u6709\u7528\u7684\u4fe1\u606f\u3002\u4e3a\u4e86\u8ba9\u8fd9\u4e9b\u4fe1\u606f\u5c3d\u53ef\u80fd\u7cbe\u786e\uff0c\u4efb\u4f55\u5173\u4e8e\u7ef4\u751f\u7d20\u6216\u77ff\u7269\u8d28\u7684\u68c0\u6d4b\u90fd\u5fc5\u987b\u53cd\u6620\u51fa\u8425\u517b\u7d20\u5728\u4f53\u5185\u53d1\u6325\u4f5c\u7528\u7684\u80fd\u529b\u3002\u4f8b\u5982\uff0c\u94c1\u662f\u6784\u6210\u8840\u7ea2\u7ec6\u80de\u7684\u5fc5\u9700\u5143\u7d20\uff0c\u6709\u52a9\u4e8e\u4f53\u5185\u6c27\u7684\u8fd0\u8f93\u3002\u901a\u8fc7\u6d4b\u5b9a\u7ec6\u80de\u4e2d\u94c1\u7684\u8425\u517b\u72b6\u51b5\uff0c\u53ef\u80fd\u4f1a\u5bf9\u4f60\u7684\u94c1\u9700\u6c42\u6709\u4e00\u4e2a\u5f88\u597d\u7684\u4f30\u6d4b\u3002\u4ece\u53e6\u4e00\u65b9\u9762\u6765\u8bb2\uff0c\u7ef4\u751f\u7d20\u6c0f\u5728\u8840\u6db2\u4e2d\u6ca1\u6709\u50cf\u94c1\u90a3\u6837\u7684\u76f4\u63a5\u4f5c\u7528\u3002\u5b83\u4e3b\u8981\u4f5c\u7528\u4e8e\u5927\u8111\u5f53\u4e2d\u7684\u5316\u5b66\u7269\u8d285-\u7f9f\u8272\u80fa\uff0c\u6216\u5728\u7532\u57fa\u5316\u53cd\u5e94\u4e2d\u8d77\u4f5c\u7528\uff0c\u964d\u4f4e\u540c\u578b\u534a\u80f1\u6c28\u9178\u542b\u91cf\u3002\u6240\u4ee5\uff0c\u4ec5\u4ec5\u6d4b\u5b9a\u8840\u6db2\u4e2d\u7684\u7ef4\u751f\u7d20B6,\u5e76\u4e0d\u80fd\u771f\u5b9e\u5730\u53cd\u6620\u4f60\u662f\u5426\u83b7\u5f97\u4e86\u5145\u8db3\u7684\u8425\u517b\u7d20\u4f9b\u7ed9\u4ee5\u53ca\u4f53\u5185\u7684\u7ef4\u751f\u7d20B6\u662f\u5426\u8fd0\u8f6c\u6b63\u5e38\u3002\u76f8\u6bd4\u4e4b\u4e0b\uff0c\u68c0\u6d4b\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u7b49\u9879\u76ee\u610f\u4e49\u66f4\u5927\u3002\u5982\u679c\u4f60\u5df2\u7ecf\u83b7\u5f97\u4e86\u5145\u8db3\u7684\u7ef4\u751f\u7d20\u3001\u7ef4\u751f\u7d20B\u548c\u53f6\u9178\uff0c\u90a3\u4e48\u4f60\u7684\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u5c31\u5f88\u4f4e\uff0c\u4f46\u5047\u5982\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u5f88\u9ad8\uff0c\u5219\u8868\u660e\u4f60\u9700\u8981\u66f4\u591a\u7684\u5176\u4e2d\u4e00\u79cd\u6216\u591a\u79cd\u8425\u517b\u7d20\u3002\u6211\u4eec\u79f0\u8fd9\u79cd\u6d4b\u8bd5\u65b9\u5f0f\u4e3a\u529f\u80fd\u6d4b\u8bd5\uff0c\u56e0\u4e3a\u8fd9\u4e2a\u6d4b\u8bd5\u80fd\u591f\u7cbe\u786e\u5730\u6d4b\u5b9a\u4f60\u4f53\u5185\u751f\u5316\u53cd\u5e94\u7684\u67d0\u9879\u7279\u6b8a\u529f\u80fd\u662f\u5426\u8fd0\u8f6c\u6b63\u5e38\u3002\r\nstatements:\r\n1.\u81b3\u98df\u5206\u6790\u662f\u4e00\u79cd\u8bc4\u4f30\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u7684\u65b9\u6cd5\u3002\r\n2.\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u786e\u5b9a\u4eba\u4eec\u6444\u5165\u7684\u8425\u517b\u7d20\u79cd\u7c7b\u548c\u6570\u91cf\u3002\r\n3.\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63ed\u793a\u4eba\u4eec\u662f\u5426\u7f3a\u4e4f\u67d0\u4e9b\u8425\u517b\u7d20\u3002\r\n4.\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63ed\u793a\u98df\u7269\u4e2d\u662f\u5426\u5b58\u5728\u4e00\u4e9b\u5bf9\u5065\u5eb7\u6709\u5bb3\u7684\u7269\u8d28\u3002\r\n5.\u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\u7684\u8bc4\u4f30\u3002\r\n6.\u81b3\u98df\u5206\u6790\u53ef\u80fd\u5e76\u4e0d\u662f\u8bc4\u4f30\u4e2a\u4eba\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u6700\u51c6\u786e\u7684\u65b9\u6cd5\u3002\r\nAnswer:\r\n<\/code><\/pre>\n<p>\u5f97\u5230\u7ed3\u679c2\uff1a<\/p>\n<pre><code>1. \u81b3\u98df\u5206\u6790\u662f\u4e00\u79cd\u8bc4\u4f30\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u7684\u65b9\u6cd5.\r\nexplanation: the context mentions that dietary analysis is a method to understand what food people consume. verdict: yes.\r\n2. \u81b3\u98df\u5206\u6790\u53ef\u4ee5\u786e\u5b9a\u4eba\u4eec\u6444\u5165\u7684\u8425\u517b\u7d20\u79cd\u7c7b\u548c\u6570\u91cf.\r\nexplanation: the context mentions that dietary analysis can evaluate the intake of nutrients such as sugar, salt, coffee, tea, alcohol, food additives, and preservatives. it does not explicitly state that it can determine the types and quantities of nutrients consumed. verdict: no.\r\n3. \u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63ed\u793a\u4eba\u4eec\u662f\u5426\u7f3a\u4e4f\u67d0\u4e9b\u8425\u517b\u7d20.\r\nexplanation: the context mentions that dietary analysis can be useful in assessing the intake of nutrients that can affect people's nutritional needs. this implies that it can reveal if someone is lacking certain nutrients. verdict: yes.\r\n4. \u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63ed\u793a\u98df\u7269\u4e2d\u662f\u5426\u5b58\u5728\u4e00\u4e9b\u5bf9\u5065\u5eb7\u6709\u5bb3\u7684\u7269\u8d28.\r\nexplanation: the context mentions that dietary analysis can be useful in evaluating the intake of substances that can affect people's nutritional needs, such as food additives and preservatives. this implies that it can reveal the presence of harmful substances in food. verdict: yes.\r\n5. \u81b3\u98df\u5206\u6790\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\u7684\u8bc4\u4f30.\r\nexplanation: the context mentions that the intake of factors such as fat, carbohydrates, protein, and calories can be evaluated through dietary analysis. verdict: yes.\r\n6. \u81b3\u98df\u5206\u6790\u53ef\u80fd\u5e76\u4e0d\u662f\u8bc4\u4f30\u4e2a\u4eba\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u6700\u51c6\u786e\u7684\u65b9\u6cd5.\r\nexplanation: the context mentions that while dietary analysis can provide useful information, it may not be the most accurate method for assessing an individual's exact nutritional status. verdict: yes.\r\nfinal verdict for each statement in order: yes. no. yes. yes. yes. yes.\r\n<\/code><\/pre>\n<p>\u6839\u636e<code>final verdict for each statement in order:<\/code>\u5b9a\u4f4d\u5230<code>yes. no. yes. yes. yes. yes.<\/code><\/p>\n<p>\u7b97\u5206\uff1a<\/p>\n<pre><code>output = \"yes. no. yes. yes. yes. yes.\"\r\nscore = sum(0 if \"yes\" in answer else 1\r\nfor answer in output.strip().split(\".\")\r\nif answer != \"\"\r\n)\r\n# score = 1 - score\/len(statements)\r\nsocre = 1 - 1\/6\r\n# score = 0.8333333333333334\r\n<\/code><\/pre>\n<p>\u5373\u4e3afaithfulnessd\u7684\u6700\u540e\u5f97\u5206\u3002<\/p>\n<h3>answer_relevancy<\/h3>\n<p>\u8861\u91cf\u7b54\u6848\u4e0e\u95ee\u9898\u7684\u76f8\u5173\u6027\u548c\u91cd\u70b9\u3002<\/p>\n<p>\u5bf9\u4e8e\u7ed9\u5b9a\u7684\u751f\u6210\u7b54\u6848\uff0cRagas \u4f7f\u7528 LLM \u627e\u51fa\u751f\u6210\u7684\u7b54\u6848\u5c06\u6210\u4e3a\u7b54\u6848\u7684\u53ef\u80fd\u95ee\u9898\uff0c\u5e76\u8ba1\u7b97\u4e0e\u5b9e\u9645\u95ee\u9898\u7684\u76f8\u4f3c\u6027\u3002<\/p>\n<p>\u6839\u636eanswer:<\/p>\n<pre><code>Generate question for the given answer.\r\nAnswer:\r\nThe PSLV-C56 mission is scheduled to be launched on Sunday, 30 July 2023 at 06:30 IST \/ 01:00 UTC. It will be launched from the Satish Dhawan Space Centre, Sriharikota, Andhra Pradesh, India \r\nQuestion: When is the scheduled launch date and time for the PSLV-C56 mission, and where will it be launched from?\r\nAnswer:\u81b3\u98df\u5206\u6790\u662f\u4e00\u79cd\u8bc4\u4f30\u4e2a\u4eba\u996e\u98df\u4e2d\u6444\u5165\u7684\u98df\u7269\u79cd\u7c7b\u548c\u6570\u91cf\u7684\u65b9\u6cd5\u3002\u901a\u8fc7\u5bf9\u4e00\u5468\u5185\u6240\u5403\u7684\u98df\u7269\u8fdb\u884c\u8be6\u7ec6\u5206\u6790\uff0c\u53ef\u4ee5\u4e86\u89e3\u4eba\u4eec\u6444\u5165\u7684\u8425\u517b\u7d20\u79cd\u7c7b\u548c\u542b\u91cf\uff0c\u4ece\u800c\u53d1\u73b0\u53ef\u80fd\u5b58\u5728\u7684\u8425\u517b\u7d20\u7f3a\u4e4f\u6216\u8fc7\u91cf\u7684\u60c5\u51b5\u3002\u81b3\u98df\u5206\u6790\u5bf9\u4e8e\u8bc4\u4f30\u98df\u7269\u4e2d\u7684\u8425\u517b\u7d20\u542b\u91cf\u3001\u6444\u5165\u91cf\u4ee5\u53ca\u4e00\u4e9b\u5f71\u54cd\u8425\u517b\u9700\u6c42\u7684\u56e0\u7d20\uff08\u5982\u7cd6\u3001\u76d0\u3001\u5496\u5561\u56e0\u3001\u9152\u7cbe\u7b49\uff09\u662f\u6709\u7528\u7684\u3002\u7136\u800c\uff0c\u81b3\u98df\u5206\u6790\u5e76\u4e0d\u80fd\u7efc\u5408\u8003\u8651\u5230\u4e2a\u4f53\u7684\u8425\u517b\u7d20\u9700\u6c42\u3001\u8425\u517b\u7d20\u5438\u6536\u548c\u5229\u7528\u7684\u60c5\u51b5\uff0c\u56e0\u6b64\u5e76\u4e0d\u662f\u8bc4\u4f30\u4e2a\u4eba\u8425\u517b\u72b6\u51b5\u7684\u552f\u4e00\u65b9\u6cd5\u3002\r\nQuestion:\r\n<\/code><\/pre>\n<p>\u5f97\u5230\uff1a<\/p>\n<pre><code>[\r\n\"What is dietary analysis and how is it used to evaluate an individual's nutritional intake?\", \r\n\"What is dietary analysis and how is it used to assess an individual's nutritional intake?\", \r\n\"What is dietary analysis and how is it used to assess an individual's nutritional intake?\"\r\n]\r\n<\/code><\/pre>\n<p>\u7136\u540e\u8ba1\u7b97[\u95ee\u9898]\u548c\u8fd9\u4e09\u4e2a\u7ed3\u679c\u7684\u76f8\u4f3c\u5ea6\uff1a<\/p>\n<p>\u76f8\u4f3c\u5ea6\u8ba1\u7b97\u8fc7\u7a0b\uff1a\u5229\u7528<code>text-embedding-ada-002<\/code>\u8fdb\u884c\u5411\u91cf\u5316\uff0c\u7136\u540e\u8ba1\u7b97\u4f59\u5f26\u76f8\u4f3c\u5ea6\u3002<\/p>\n<pre><code>def calculate_similarity(\r\nself: t.Self, question: str, generated_questions: list[str]\r\n):\r\nquestion_vec = np.asarray(self.embedding.embed_query(question)).reshape(1, -1)\r\ngen_question_vec = np.asarray(\r\nself.embedding.embed_documents(generated_questions)\r\n)\r\nnorm = np.linalg.norm(gen_question_vec, axis=1) * np.linalg.norm(\r\nquestion_vec, axis=1)\r\nreturn (\r\nnp.dot(gen_question_vec, question_vec.T).reshape(-1,)\/ norm\r\n)\r\n<\/code><\/pre>\n<p>\u5f97\u5230\uff1a<\/p>\n<pre><code>[0.83663467 0.83484782 0.83484782]\r\n<\/code><\/pre>\n<p>\u7136\u540e\u5e73\u5747\uff1a<\/p>\n<pre><code>answer_relevancy_score: 0.8354434364200779\r\n<\/code><\/pre>\n<h3>context_relevancy<\/h3>\n<p>\u6d4b\u91cf\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u4e2d\u7684\u4fe1\u566a\u6bd4\u3002\u7ed9\u5b9a\u4e00\u4e2a\u95ee\u9898\uff0cRagas\u6253\u7535\u8bdd\u7ed9LLM\uff0c\u4ece\u68c0\u7d22\u5230\u7684\u4e0a\u4e0b\u6587\u4e2d\u627e\u51fa\u56de\u7b54\u95ee\u9898\u6240\u9700\u7684\u53e5\u5b50\u3002\u6240\u9700\u53e5\u5b50\u4e0e\u4e0a\u4e0b\u6587\u4e2d\u603b\u53e5\u5b50\u4e4b\u95f4\u7684\u6bd4\u7387\u4e3a\u60a8\u63d0\u4f9b\u5206\u6570\u3002<\/p>\n<p>\u8f93\u5165\uff1a<\/p>\n<pre><code>Task: Candidate sentence extraction.\r\nGiven the question and context, extract minimum number of sentences from context required to answer the question. If the context do not contain information required to answer the question return \"No candidate sentences found\".\r\nquestion: Which equation is known as worlds most famous equation?\r\ncontext:\r\nAlbert Einstein (14 March 1879 \u2013 18 April 1955) was a German-born theoretical physicist,[5] widely ranked among the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century.\r\nHis mass\u2013energy equivalence formula E = mc2, which arises from relativity theory, has been called \"the world's most famous equation\".\r\nsentences:His mass\u2013energy equivalence formula E = mc2, which arises from relativity theory, has been called \"the world's most famous equation\".\r\nquestion: Were Scott Derrickson and Ed Wood of the same nationality?\r\ncontext :\r\nScott Derrickson (born July 16, 1966) is an American director, screenwriter and producer He lives in Los Angeles, California He is best known for directing horror films such as \"Sinister\", \"The Exorcism of Emily Rose\", and \"Deliver Us From Evil\", as well as the 2016 Marvel Cinematic Universe installment, \"Doctor Strange\"Tyler Bates is an American musician, music producer, and composer for films, television, and video games. Adam Collis is an American filmmaker and actor.Conrad Brooks is an American actor.Edward Davis Wood Jr. (October 10, 1924 \u2013 December 10, 1978) was an American filmmaker, actor, writer, producer, and director.\r\nsentences:Scott Derrickson (born July 16, 1966) is an American director, screenwriter and producer. Edward Davis Wood Jr. (October 10, 1924 \u2013 December 10, 1978) was an American filmmaker, actor, writer, producer, and director.\r\nquestion: How many were killed in the Tiananmen Square incident?\r\ncontext:\r\nTiananmen Square incident, also called June Fourth incident or 6\/4, series of protests and demonstrations in China in the spring of 1989 that culminated on the night of June 3\u20134 with a government crackdown on the demonstrators in Tiananmen Square in Beijing.\r\nsentences: No candidate sentences found.\r\nquestion:\u81b3\u98df\u5206\u6790\u662f\u4ec0\u4e48?\r\ncontext:\r\n\u4f60\u60f3\u8981\u8fbe\u5230\u4ec0\u4e48\u6837\u7684\u5065\u5eb7\u7a0b\u5ea6\uff1f\u5982\u679c\u4f60\u60f3\u4e86\u89e3\u81ea\u5df1\u7cbe\u795e\u548c\u8eab\u4f53\u7684\u5168\u90e8\u6f5c\u5728\u80fd\u529b\uff0c\u786e\u5b9a\u4f60\u7684\u6700\u4f73\u8425\u517b\u9700\u6c42\u662f\u975e\u5e38\u5fc5\u8981\u7684\u3002\u4f46\u662f\u5982\u679c\u4f60\u7684\u8425\u517b\u9700\u6c42\u5f88\u7279\u522b\uff0c\u90a3\u4e48\u4f60\u8be5\u5982\u4f55\u53d1\u73b0\u8fd9\u4e00\u70b9\u5462\uff1f\u4ece1980\u5e74\u5f00\u59cb\uff0c\u6211\u5236\u5b9a\u5e76\u5b8c\u5584\u4e86\u4e00\u5957\u5206\u6790\u4eba\u4f53\u8425\u517b\u9700\u6c42\u7684\u4f53\u7cfb\uff0c\u6b64\u4f53\u7cfb\u5efa\u7acb\u5728\u8bc4\u4f30\u5f71\u54cd\u4e2a\u4eba\u8425\u517b\u9700\u6c42\u7684\u4e3b\u8981\u56e0\u7d20\u7684\u57fa\u7840\u4e0a\u3002\u6b64\u4f53\u7cfb\u5df2\u7ecf\u572810\u4e07\u4eba\u5f53\u4e2d\u8fdb\u884c\u4e86\u6d4b\u8bd5\u5e76\u5f97\u5230\u4e86\u9a8c\u8bc1\uff0c\u73b0\u5728\u5df2\u88ab\u4e16\u754c\u5404\u56fd\u7684\u4e34\u5e8a\u8425\u517b\u5b66\u5bb6\u6240\u5e94\u7528\u3002\u65e0\u6570\u4eba\u5df2\u7ecf\u4ece\u4e2d\u83b7\u76ca\uff0c\u56e0\u6b64\uff0c\u6211\u77e5\u9053\u4ece\u8fd9\u4e2a\u4f53\u7cfb\u4e2d\u53ef\u4ee5\u5f97\u5230\u4ec0\u4e48\u6837\u7684\u7ed3\u679c\u3002\u8fd9\u4e9b\u7ed3\u679c\u5305\u62ec\uff1a\u66f4\u654f\u9510\u7684\u601d\u7ef4\u3001\u66f4\u597d\u7684\u8bb0\u5fc6\u529b\u3001\u66f4\u5f3a\u7684\u4f53\u529b\u3001\u66f4\u597d\u7684\u4f53\u91cd\u63a7\u5236\u6548\u679c\u3001\u964d\u4f4e\u80c6\u56fa\u9187\u542b\u91cf\u3001\u4f7f\u5f97\u75c5\u60c5\u597d\u8f6c\u3002\u5c3d\u7ba1\u591a\u6570\u88ab\u8bca\u65ad\u4e3a\u6709\u75c5\u7684\u4eba\u5df2\u7ecf\u53d7\u76ca\u4e8e\u4e2a\u4eba\u5065\u5eb7\u8ba1\u5212\uff0c\u4f46\u8fd9\u4e2a\u5065\u5eb7\u8ba1\u5212\u662f\u4ee5\u9884\u9632\u4e3a\u4e3b\u8981\u76ee\u6807\uff0c\u800c\u4e0d\u662f\u4ee5\u6cbb\u75c5\u4e3a\u4e3b\u7684\u3002\u5982\u679c\u4f60\u6b63\u5728\u8fdb\u884c\u533b\u836f\u6cbb\u7597\uff0c\u8bf7\u786e\u8ba4\u8fd9\u4e2a\u8425\u517b\u8865\u5145\u65b9\u6848\u4e0e\u4f60\u63a5\u53d7\u7684\u6cbb\u7597\u63aa\u65bd\u662f\u5426\u80fd\u591f\u5f88\u597d\u5730\u76f8\u5bb9\u3002\u5f71\u54cd\u4f60\u8425\u517b\u9700\u6c42\u7684\u56e0\u7d20\uff0c\u81f3\u5c11\u67098\u9879\u56e0\u7d20\u4f1a\u5f71\u54cd\u4f60\u7684\u8425\u517b\u9700\u6c42\u91cf\u3002\u5e74\u9f84\u3001\u6027\u522b\u4ee5\u53ca\u8fd0\u52a8\u91cf\u90fd\u662f\u5f88\u5e38\u89c1\u7684\u5f71\u54cd\u56e0\u7d20\uff0c\u4f46\u6c61\u67d3\u3001\u538b\u529b\u3001\u57fa\u56e0\u9057\u4f20\u3001\u4f60\u8fc7\u53bb\u7684\u5065\u5eb7\u72b6\u51b5\uff0c\u5f53\u7136\u8fd8\u6709\u81b3\u98df\u6240\u80fd\u63d0\u4f9b\u7684\u8425\u517b\u7d20\u4ee5\u53ca\u53cd\u8425\u517b\u7269\u8d28\uff0c\u5374\u4e0d\u662f\u80fd\u591f\u8f7b\u6613\u4e86\u89e3\u7684\u3002\u4f46\u662f\u6240\u6709\u7684\u8fd9\u4e9b\u7ec6\u8282\u4ee5\u53ca\u66f4\u591a\u7684\u56e0\u7d20\uff0c\u90fd\u5fc5\u987b\u7eb3\u5165\u8003\u8651\u5f53\u4e2d\u3002\u901a\u8fc74\u79cd\u5206\u6790\u65b9\u6cd5\u2014\u2014\u81b3\u98df\u5206\u6790\u3001\u751f\u5316\u5206\u6790\u3001\u75c7\u72b6\u5206\u6790\u3001\u751f\u6d3b\u65b9\u5f0f\u5206\u6790\uff0c\u53ef\u4ee5\u4e86\u89e3\u8fd9\u4e9b\u7ec6\u8282\u3002\u81b3\u98df\u5206\u6790\u8fd9\u79cd\u65b9\u5f0f\u4f3c\u4e4e\u662f\u4ece\u5bb9\u6613\u7740\u624b\u7684\u5730\u65b9\u5f00\u59cb\uff1a\u641e\u6e05\u695a\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u662f\u4ec0\u4e48\uff0c\u5c31\u80fd\u53d1\u73b0\u7f3a\u4e4f\u7684\u8425\u517b\u7d20\u662f\u4ec0\u4e48\u3002\r\n\u4f46\u4e0d\u5e78\u7684\u662f\uff0c\u5982\u679c\u5bf9\u4e00\u5468\u5185\u6240\u5403\u7684\u98df\u7269\u8fdb\u884c\u8be6\u7ec6\u5206\u6790\uff0c\u5e76\u4e0d\u80fd\u7efc\u5408\u8003\u8651\u5230\u98df\u7269\u8425\u517b\u7d20\u542b\u91cf\u7684\u53d8\u5f02\u6027\u3001\u4e2a\u4eba\u7684\u8425\u517b\u7d20\u9700\u6c42\uff0c\u6216\u8005\u8425\u517b\u7d20\u662f\u5426\u80fd\u88ab\u5438\u6536\u3001\u5438\u6536\u540e\u7684\u8425\u517b\u7d20\u53c8\u662f\u5426\u80fd\u88ab\u6709\u6548\u5730\u5229\u7528\u7b49\u60c5\u51b5\u3002\u6211\u89c1\u8fc7\u8bb8\u591a\u4eba\u7684\u996e\u98df\u8d28\u91cf\u5f88\u9ad8\uff0c\u5374\u4f9d\u7136\u6709\u7ef4\u751f\u7d20\u7f3a\u4e4f\u7684\u75c7\u72b6\u3002\u5bf9\u5176\u4e2d\u7edd\u5927\u90e8\u5206\u4eba\u6765\u8bf4\uff0c\u95ee\u9898\u5728\u4e8e\u8425\u517b\u7d20\u5438\u6536\u4e0d\u826f\u3002\u8fd9\u4e9b\u53ef\u53d8\u56e0\u7d20\uff0c\u4f7f\u5f97\u67d0\u4e9b\u901a\u8fc7\u8ba1\u7b97\u673a\u8fdb\u884c\u7684\u81b3\u98df\u5206\u6790\u4e0d\u5982\u8bbe\u60f3\u7684\u51c6\u786e\u3002\u81b3\u98df\u5206\u6790\u5bf9\u4e8e\u8bc4\u4f30\u90a3\u4e9b\u80fd\u5f71\u54cd\u5230\u4eba\u4eec\u8425\u517b\u7d20\u9700\u6c42\u7684\u98df\u7269\u5f88\u6709\u7528\u5904\uff0c\u6bd4\u5982\u7cd6\u3001\u76d0\u3001\u5496\u5561\u3001\u8336\u3001\u9152\u7cbe\u3001\u98df\u54c1\u6dfb\u52a0\u5242\u548c\u9632\u8150\u5242\u3002\u53e6\u4e00\u4e9b\u56e0\u7d20\uff0c\u5982\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\uff0c\u4e5f\u53ef\u7531\u81b3\u98df\u5206\u6790\u8fdb\u884c\u8bc4\u4f30\u3002\u8bf8\u5982\u5934\u53d1\u77ff\u7269\u8d28\u5206\u6790\u4e0e\u7ef4\u751f\u7d20\u8840\u6db2\u5206\u6790\u8fd9\u6837\u7684\u5b9e\u9a8c,\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u4f53\u5185\u751f\u7269\u5316\u5b66\u72b6\u6001\u7684\u786e\u5207\u4fe1\u606f\uff0c\u4e5f\u80fd\u8ba9\u8425\u517b\u987e\u95ee\u4e86\u89e3\u4f60\u8eab\u4f53\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u3002\u4f46\u5e76\u4e0d\u662f\u6240\u6709\u8fd9\u4e9b\u5b9e\u9a8c\u90fd\u80fd\u5bf9\u89c4\u5212\u4f60\u7684\u8425\u517b\u65b9\u6848\u63d0\u4f9b\u6709\u7528\u7684\u4fe1\u606f\u3002\u4e3a\u4e86\u8ba9\u8fd9\u4e9b\u4fe1\u606f\u5c3d\u53ef\u80fd\u7cbe\u786e\uff0c\u4efb\u4f55\u5173\u4e8e\u7ef4\u751f\u7d20\u6216\u77ff\u7269\u8d28\u7684\u68c0\u6d4b\u90fd\u5fc5\u987b\u53cd\u6620\u51fa\u8425\u517b\u7d20\u5728\u4f53\u5185\u53d1\u6325\u4f5c\u7528\u7684\u80fd\u529b\u3002\u4f8b\u5982\uff0c\u94c1\u662f\u6784\u6210\u8840\u7ea2\u7ec6\u80de\u7684\u5fc5\u9700\u5143\u7d20\uff0c\u6709\u52a9\u4e8e\u4f53\u5185\u6c27\u7684\u8fd0\u8f93\u3002\u901a\u8fc7\u6d4b\u5b9a\u7ec6\u80de\u4e2d\u94c1\u7684\u8425\u517b\u72b6\u51b5\uff0c\u53ef\u80fd\u4f1a\u5bf9\u4f60\u7684\u94c1\u9700\u6c42\u6709\u4e00\u4e2a\u5f88\u597d\u7684\u4f30\u6d4b\u3002\u4ece\u53e6\u4e00\u65b9\u9762\u6765\u8bb2\uff0c\u7ef4\u751f\u7d20\u6c0f\u5728\u8840\u6db2\u4e2d\u6ca1\u6709\u50cf\u94c1\u90a3\u6837\u7684\u76f4\u63a5\u4f5c\u7528\u3002\u5b83\u4e3b\u8981\u4f5c\u7528\u4e8e\u5927\u8111\u5f53\u4e2d\u7684\u5316\u5b66\u7269\u8d285-\u7f9f\u8272\u80fa\uff0c\u6216\u5728\u7532\u57fa\u5316\u53cd\u5e94\u4e2d\u8d77\u4f5c\u7528\uff0c\u964d\u4f4e\u540c\u578b\u534a\u80f1\u6c28\u9178\u542b\u91cf\u3002\u6240\u4ee5\uff0c\u4ec5\u4ec5\u6d4b\u5b9a\u8840\u6db2\u4e2d\u7684\u7ef4\u751f\u7d20B6,\u5e76\u4e0d\u80fd\u771f\u5b9e\u5730\u53cd\u6620\u4f60\u662f\u5426\u83b7\u5f97\u4e86\u5145\u8db3\u7684\u8425\u517b\u7d20\u4f9b\u7ed9\u4ee5\u53ca\u4f53\u5185\u7684\u7ef4\u751f\u7d20B6\u662f\u5426\u8fd0\u8f6c\u6b63\u5e38\u3002\u76f8\u6bd4\u4e4b\u4e0b\uff0c\u68c0\u6d4b\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u7b49\u9879\u76ee\u610f\u4e49\u66f4\u5927\u3002\u5982\u679c\u4f60\u5df2\u7ecf\u83b7\u5f97\u4e86\u5145\u8db3\u7684\u7ef4\u751f\u7d20\u3001\u7ef4\u751f\u7d20B\u548c\u53f6\u9178\uff0c\u90a3\u4e48\u4f60\u7684\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u5c31\u5f88\u4f4e\uff0c\u4f46\u5047\u5982\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u5f88\u9ad8\uff0c\u5219\u8868\u660e\u4f60\u9700\u8981\u66f4\u591a\u7684\u5176\u4e2d\u4e00\u79cd\u6216\u591a\u79cd\u8425\u517b\u7d20\u3002\u6211\u4eec\u79f0\u8fd9\u79cd\u6d4b\u8bd5\u65b9\u5f0f\u4e3a\u529f\u80fd\u6d4b\u8bd5\uff0c\u56e0\u4e3a\u8fd9\u4e2a\u6d4b\u8bd5\u80fd\u591f\u7cbe\u786e\u5730\u6d4b\u5b9a\u4f60\u4f53\u5185\u751f\u5316\u53cd\u5e94\u7684\u67d0\u9879\u7279\u6b8a\u529f\u80fd\u662f\u5426\u8fd0\u8f6c\u6b63\u5e38\u3002\r\nsentences:\r\n<\/code><\/pre>\n<p>\u5f97\u5230\uff1a<\/p>\n<pre><code>[\r\n'\u81b3\u98df\u5206\u6790\u8fd9\u79cd\u65b9\u5f0f\u4f3c\u4e4e\u662f\u4ece\u5bb9\u6613\u7740\u624b\u7684\u5730\u65b9\u5f00\u59cb\uff1a\u641e\u6e05\u695a\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u662f\u4ec0\u4e48\uff0c\u5c31\u80fd\u53d1\u73b0\u7f3a\u4e4f\u7684\u8425\u517b\u7d20\u662f\u4ec0\u4e48\u3002', \r\n'\u81b3\u98df\u5206\u6790\u662f\u4ece\u5bb9\u6613\u7740\u624b\u7684\u5730\u65b9\u5f00\u59cb\uff1a\u641e\u6e05\u695a\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u662f\u4ec0\u4e48\uff0c\u5c31\u80fd\u53d1\u73b0\u7f3a\u4e4f\u7684\u8425\u517b\u7d20\u662f\u4ec0\u4e48\u3002'\r\n]\r\n<\/code><\/pre>\n<p>\u5206\u5272\u6bcf\u4e2a\u7ed3\u679c\uff0c\u5728\u4e0a\u4e0b\u6587\u4e2d\u67e5\u627e\uff0c\u770b\u6bcf\u4e2a\u7ed3\u679c\u4e2d\u5206\u5272\u540e\u7684\u5c0f\u53e5\u5728\u539f\u6587\u4e2d\u51fa\u73b0\u7684\u5360\u6bd4overlap_scores\u3002<br \/>\n\u5047\u8bbe\u5f97\u5230<code>[1.0, 1.0]<\/code>\u3002<\/p>\n<p>\u4e0a\u4e0b\u6587\uff1a<\/p>\n<pre><code>\u4f60\u60f3\u8981\u8fbe\u5230\u4ec0\u4e48\u6837\u7684\u5065\u5eb7\u7a0b\u5ea6\uff1f\u5982\u679c\u4f60\u60f3\u4e86\u89e3\u81ea\u5df1\u7cbe\u795e\u548c\u8eab\u4f53\u7684\u5168\u90e8\u6f5c\u5728\u80fd\u529b\uff0c\u786e\u5b9a\u4f60\u7684\u6700\u4f73\u8425\u517b\u9700\u6c42\u662f\u975e\u5e38\u5fc5\u8981\u7684\u3002\u4f46\u662f\u5982\u679c\u4f60\u7684\u8425\u517b\u9700\u6c42\u5f88\u7279\u522b\uff0c\u90a3\u4e48\u4f60\u8be5\u5982\u4f55\u53d1\u73b0\u8fd9\u4e00\u70b9\u5462\uff1f\u4ece1980\u5e74\u5f00\u59cb\uff0c\u6211\u5236\u5b9a\u5e76\u5b8c\u5584\u4e86\u4e00\u5957\u5206\u6790\u4eba\u4f53\u8425\u517b\u9700\u6c42\u7684\u4f53\u7cfb\uff0c\u6b64\u4f53\u7cfb\u5efa\u7acb\u5728\u8bc4\u4f30\u5f71\u54cd\u4e2a\u4eba\u8425\u517b\u9700\u6c42\u7684\u4e3b\u8981\u56e0\u7d20\u7684\u57fa\u7840\u4e0a\u3002\u6b64\u4f53\u7cfb\u5df2\u7ecf\u572810\u4e07\u4eba\u5f53\u4e2d\u8fdb\u884c\u4e86\u6d4b\u8bd5\u5e76\u5f97\u5230\u4e86\u9a8c\u8bc1\uff0c\u73b0\u5728\u5df2\u88ab\u4e16\u754c\u5404\u56fd\u7684\u4e34\u5e8a\u8425\u517b\u5b66\u5bb6\u6240\u5e94\u7528\u3002\u65e0\u6570\u4eba\u5df2\u7ecf\u4ece\u4e2d\u83b7\u76ca\uff0c\u56e0\u6b64\uff0c\u6211\u77e5\u9053\u4ece\u8fd9\u4e2a\u4f53\u7cfb\u4e2d\u53ef\u4ee5\u5f97\u5230\u4ec0\u4e48\u6837\u7684\u7ed3\u679c\u3002\u8fd9\u4e9b\u7ed3\u679c\u5305\u62ec\uff1a\u66f4\u654f\u9510\u7684\u601d\u7ef4\u3001\u66f4\u597d\u7684\u8bb0\u5fc6\u529b\u3001\u66f4\u5f3a\u7684\u4f53\u529b\u3001\u66f4\u597d\u7684\u4f53\u91cd\u63a7\u5236\u6548\u679c\u3001\u964d\u4f4e\u80c6\u56fa\u9187\u542b\u91cf\u3001\u4f7f\u5f97\u75c5\u60c5\u597d\u8f6c\u3002\u5c3d\u7ba1\u591a\u6570\u88ab\u8bca\u65ad\u4e3a\u6709\u75c5\u7684\u4eba\u5df2\u7ecf\u53d7\u76ca\u4e8e\u4e2a\u4eba\u5065\u5eb7\u8ba1\u5212\uff0c\u4f46\u8fd9\u4e2a\u5065\u5eb7\u8ba1\u5212\u662f\u4ee5\u9884\u9632\u4e3a\u4e3b\u8981\u76ee\u6807\uff0c\u800c\u4e0d\u662f\u4ee5\u6cbb\u75c5\u4e3a\u4e3b\u7684\u3002\u5982\u679c\u4f60\u6b63\u5728\u8fdb\u884c\u533b\u836f\u6cbb\u7597\uff0c\u8bf7\u786e\u8ba4\u8fd9\u4e2a\u8425\u517b\u8865\u5145\u65b9\u6848\u4e0e\u4f60\u63a5\u53d7\u7684\u6cbb\u7597\u63aa\u65bd\u662f\u5426\u80fd\u591f\u5f88\u597d\u5730\u76f8\u5bb9\u3002\u5f71\u54cd\u4f60\u8425\u517b\u9700\u6c42\u7684\u56e0\u7d20\uff0c\u81f3\u5c11\u67098\u9879\u56e0\u7d20\u4f1a\u5f71\u54cd\u4f60\u7684\u8425\u517b\u9700\u6c42\u91cf\u3002\u5e74\u9f84\u3001\u6027\u522b\u4ee5\u53ca\u8fd0\u52a8\u91cf\u90fd\u662f\u5f88\u5e38\u89c1\u7684\u5f71\u54cd\u56e0\u7d20\uff0c\u4f46\u6c61\u67d3\u3001\u538b\u529b\u3001\u57fa\u56e0\u9057\u4f20\u3001\u4f60\u8fc7\u53bb\u7684\u5065\u5eb7\u72b6\u51b5\uff0c\u5f53\u7136\u8fd8\u6709\u81b3\u98df\u6240\u80fd\u63d0\u4f9b\u7684\u8425\u517b\u7d20\u4ee5\u53ca\u53cd\u8425\u517b\u7269\u8d28\uff0c\u5374\u4e0d\u662f\u80fd\u591f\u8f7b\u6613\u4e86\u89e3\u7684\u3002\u4f46\u662f\u6240\u6709\u7684\u8fd9\u4e9b\u7ec6\u8282\u4ee5\u53ca\u66f4\u591a\u7684\u56e0\u7d20\uff0c\u90fd\u5fc5\u987b\u7eb3\u5165\u8003\u8651\u5f53\u4e2d\u3002\u901a\u8fc74\u79cd\u5206\u6790\u65b9\u6cd5\u2014\u2014\u81b3\u98df\u5206\u6790\u3001\u751f\u5316\u5206\u6790\u3001\u75c7\u72b6\u5206\u6790\u3001\u751f\u6d3b\u65b9\u5f0f\u5206\u6790\uff0c\u53ef\u4ee5\u4e86\u89e3\u8fd9\u4e9b\u7ec6\u8282\u3002\u81b3\u98df\u5206\u6790\u8fd9\u79cd\u65b9\u5f0f\u4f3c\u4e4e\u662f\u4ece\u5bb9\u6613\u7740\u624b\u7684\u5730\u65b9\u5f00\u59cb\uff1a\u641e\u6e05\u695a\u4eba\u4eec\u6444\u5165\u7684\u98df\u7269\u662f\u4ec0\u4e48\uff0c\u5c31\u80fd\u53d1\u73b0\u7f3a\u4e4f\u7684\u8425\u517b\u7d20\u662f\u4ec0\u4e48\u3002\r\n\u4f46\u4e0d\u5e78\u7684\u662f\uff0c\u5982\u679c\u5bf9\u4e00\u5468\u5185\u6240\u5403\u7684\u98df\u7269\u8fdb\u884c\u8be6\u7ec6\u5206\u6790\uff0c\u5e76\u4e0d\u80fd\u7efc\u5408\u8003\u8651\u5230\u98df\u7269\u8425\u517b\u7d20\u542b\u91cf\u7684\u53d8\u5f02\u6027\u3001\u4e2a\u4eba\u7684\u8425\u517b\u7d20\u9700\u6c42\uff0c\u6216\u8005\u8425\u517b\u7d20\u662f\u5426\u80fd\u88ab\u5438\u6536\u3001\u5438\u6536\u540e\u7684\u8425\u517b\u7d20\u53c8\u662f\u5426\u80fd\u88ab\u6709\u6548\u5730\u5229\u7528\u7b49\u60c5\u51b5\u3002\u6211\u89c1\u8fc7\u8bb8\u591a\u4eba\u7684\u996e\u98df\u8d28\u91cf\u5f88\u9ad8\uff0c\u5374\u4f9d\u7136\u6709\u7ef4\u751f\u7d20\u7f3a\u4e4f\u7684\u75c7\u72b6\u3002\u5bf9\u5176\u4e2d\u7edd\u5927\u90e8\u5206\u4eba\u6765\u8bf4\uff0c\u95ee\u9898\u5728\u4e8e\u8425\u517b\u7d20\u5438\u6536\u4e0d\u826f\u3002\u8fd9\u4e9b\u53ef\u53d8\u56e0\u7d20\uff0c\u4f7f\u5f97\u67d0\u4e9b\u901a\u8fc7\u8ba1\u7b97\u673a\u8fdb\u884c\u7684\u81b3\u98df\u5206\u6790\u4e0d\u5982\u8bbe\u60f3\u7684\u51c6\u786e\u3002\u81b3\u98df\u5206\u6790\u5bf9\u4e8e\u8bc4\u4f30\u90a3\u4e9b\u80fd\u5f71\u54cd\u5230\u4eba\u4eec\u8425\u517b\u7d20\u9700\u6c42\u7684\u98df\u7269\u5f88\u6709\u7528\u5904\uff0c\u6bd4\u5982\u7cd6\u3001\u76d0\u3001\u5496\u5561\u3001\u8336\u3001\u9152\u7cbe\u3001\u98df\u54c1\u6dfb\u52a0\u5242\u548c\u9632\u8150\u5242\u3002\u53e6\u4e00\u4e9b\u56e0\u7d20\uff0c\u5982\u8102\u80aa\u3001\u78b3\u6c34\u5316\u5408\u7269\u3001\u86cb\u767d\u8d28\u548c\u70ed\u91cf\u7684\u6444\u5165\u91cf\uff0c\u4e5f\u53ef\u7531\u81b3\u98df\u5206\u6790\u8fdb\u884c\u8bc4\u4f30\u3002\u8bf8\u5982\u5934\u53d1\u77ff\u7269\u8d28\u5206\u6790\u4e0e\u7ef4\u751f\u7d20\u8840\u6db2\u5206\u6790\u8fd9\u6837\u7684\u5b9e\u9a8c,\u53ef\u4ee5\u63d0\u4f9b\u5173\u4e8e\u4f53\u5185\u751f\u7269\u5316\u5b66\u72b6\u6001\u7684\u786e\u5207\u4fe1\u606f\uff0c\u4e5f\u80fd\u8ba9\u8425\u517b\u987e\u95ee\u4e86\u89e3\u4f60\u8eab\u4f53\u7684\u786e\u5207\u8425\u517b\u72b6\u51b5\u3002\u4f46\u5e76\u4e0d\u662f\u6240\u6709\u8fd9\u4e9b\u5b9e\u9a8c\u90fd\u80fd\u5bf9\u89c4\u5212\u4f60\u7684\u8425\u517b\u65b9\u6848\u63d0\u4f9b\u6709\u7528\u7684\u4fe1\u606f\u3002\u4e3a\u4e86\u8ba9\u8fd9\u4e9b\u4fe1\u606f\u5c3d\u53ef\u80fd\u7cbe\u786e\uff0c\u4efb\u4f55\u5173\u4e8e\u7ef4\u751f\u7d20\u6216\u77ff\u7269\u8d28\u7684\u68c0\u6d4b\u90fd\u5fc5\u987b\u53cd\u6620\u51fa\u8425\u517b\u7d20\u5728\u4f53\u5185\u53d1\u6325\u4f5c\u7528\u7684\u80fd\u529b\u3002\u4f8b\u5982\uff0c\u94c1\u662f\u6784\u6210\u8840\u7ea2\u7ec6\u80de\u7684\u5fc5\u9700\u5143\u7d20\uff0c\u6709\u52a9\u4e8e\u4f53\u5185\u6c27\u7684\u8fd0\u8f93\u3002\u901a\u8fc7\u6d4b\u5b9a\u7ec6\u80de\u4e2d\u94c1\u7684\u8425\u517b\u72b6\u51b5\uff0c\u53ef\u80fd\u4f1a\u5bf9\u4f60\u7684\u94c1\u9700\u6c42\u6709\u4e00\u4e2a\u5f88\u597d\u7684\u4f30\u6d4b\u3002\u4ece\u53e6\u4e00\u65b9\u9762\u6765\u8bb2\uff0c\u7ef4\u751f\u7d20\u6c0f\u5728\u8840\u6db2\u4e2d\u6ca1\u6709\u50cf\u94c1\u90a3\u6837\u7684\u76f4\u63a5\u4f5c\u7528\u3002\u5b83\u4e3b\u8981\u4f5c\u7528\u4e8e\u5927\u8111\u5f53\u4e2d\u7684\u5316\u5b66\u7269\u8d285-\u7f9f\u8272\u80fa\uff0c\u6216\u5728\u7532\u57fa\u5316\u53cd\u5e94\u4e2d\u8d77\u4f5c\u7528\uff0c\u964d\u4f4e\u540c\u578b\u534a\u80f1\u6c28\u9178\u542b\u91cf\u3002\u6240\u4ee5\uff0c\u4ec5\u4ec5\u6d4b\u5b9a\u8840\u6db2\u4e2d\u7684\u7ef4\u751f\u7d20B6,\u5e76\u4e0d\u80fd\u771f\u5b9e\u5730\u53cd\u6620\u4f60\u662f\u5426\u83b7\u5f97\u4e86\u5145\u8db3\u7684\u8425\u517b\u7d20\u4f9b\u7ed9\u4ee5\u53ca\u4f53\u5185\u7684\u7ef4\u751f\u7d20B6\u662f\u5426\u8fd0\u8f6c\u6b63\u5e38\u3002\u76f8\u6bd4\u4e4b\u4e0b\uff0c\u68c0\u6d4b\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u7b49\u9879\u76ee\u610f\u4e49\u66f4\u5927\u3002\u5982\u679c\u4f60\u5df2\u7ecf\u83b7\u5f97\u4e86\u5145\u8db3\u7684\u7ef4\u751f\u7d20\u3001\u7ef4\u751f\u7d20B\u548c\u53f6\u9178\uff0c\u90a3\u4e48\u4f60\u7684\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u5c31\u5f88\u4f4e\uff0c\u4f46\u5047\u5982\u540c\u578b\u534a\u80f1\u6c28\u9178\u6d53\u5ea6\u5f88\u9ad8\uff0c\u5219\u8868\u660e\u4f60\u9700\u8981\u66f4\u591a\u7684\u5176\u4e2d\u4e00\u79cd\u6216\u591a\u79cd\u8425\u517b\u7d20\u3002\u6211\u4eec\u79f0\u8fd9\u79cd\u6d4b\u8bd5\u65b9\u5f0f\u4e3a\u529f\u80fd\u6d4b\u8bd5\uff0c\u56e0\u4e3a\u8fd9\u4e2a\u6d4b\u8bd5\u80fd\u591f\u7cbe\u786e\u5730\u6d4b\u5b9a\u4f60\u4f53\u5185\u751f\u5316\u53cd\u5e94\u7684\u67d0\u9879\u7279\u6b8a\u529f\u80fd\u662f\u5426\u8fd0\u8f6c\u6b63\u5e38\u3002\r\n<\/code><\/pre>\n<p>\u7136\u540e\u5229\u7528bert\u6216jaccard\u8ba1\u7b97\u51fa\u4e24\u4e2a\u7b54\u6848\u7684\u76f8\u4f3c\u5ea6\u5f97\u5230agr_score\u3002<br \/>\n\u5219\u53ef\u7b97\u51facontext_relevancy\uff1a<\/p>\n<pre><code>context_relevancy = agr_score * (overlap_scores\u7684\u5e73\u5747\u503c)\r\n<\/code><\/pre>\n<h3>context_recall<\/h3>\n<p>\u5728\u6d4b\u8bd5\u8fd9\u4e2a\u4e4b\u524d\uff0c\u4f60\u9700\u8981\u63d0\u4f9b\u4e00\u4e2a<a href=\"https:\/\/github.com\/explodinggradients\/ragas\/issues\/101\">ground_truth<\/a>\u3002<br \/>\n\u6839\u636e\u63d0\u793a\uff1a<\/p>\n<pre><code>Given a context, and an answer, analyze each sentence in the answer and classify if the sentence can be attributed to the given context or not.\r\nThink in steps and reason bofore coming to conclusion. \r\ncontext: Albert Einstein (14 March 1879 \u2013 18 April 1955) was a German-born theoretical physicist,widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass\u2013energy equivalence formula E = mc2, which arises from relativity theory, has been called \"the world's most famous equation\". He received the 1921 Nobel Prize in Physics \"for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect\", a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.\r\nanswer: Albert Einstein born in 14 March 1879 was  German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics \"for his services to theoretical physics. He published 4 papers in 1905.  Einstein moved to Switzerland in 1895 \r\nclassification\r\n1. Albert Einstein born in 14 March 1879 was  German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. The date of birth of Einstein is mentioned clearly in the context. So [Attributed]\r\n2. He received the 1921 Nobel Prize in Physics \"for his services to theoretical physics. The exact sentence is present in the given context. So [Attributed]\r\n3. He published 4 papers in 1905. There is no mention about papers he wrote in given the context. So [Not Attributed]\r\n4. Einstein moved to Switzerland in 1895. There is not supporting evidence for this in the given the context. So [Not Attributed]\r\ncontext:{context}\r\nanswer:{ground_truth}\r\nclassification:\r\n<\/code><\/pre>\n<p>\u5f97\u5230\u7ed3\u679c\uff1a<\/p>\n<pre><code>[\r\n'#############[Attributed]',\r\n'#############[Attributed]',\r\n'#############[Not Attributed]',\r\n\u2026\u2026\r\n]\r\n<\/code><\/pre>\n<p>\u8ba1\u7b97[Attributed]\u51fa\u73b0\u5360\u6bd4\uff0c\u5373\u4e3acontext_recall\u3002<\/p>\n<h3>\u4e2a\u4eba\u770b\u6cd5\uff1a<\/h3>\n<p>\u5173\u4e8e\u8fd9\u4e2a\u5e93\uff1a\u4e0d\u652f\u6301auzre\u63a5\u5165\u7684openai\uff0c\u9700\u8981\u6539\u6e90\u4ee3\u7801<\/p>\n<p><strong>\u5173\u4e8e\u56db\u4e2a\u6307\u6807<\/strong><\/p>\n<p>context_relevancy\uff1a \u5b58\u7591 (agr_score\u4e2d\u8ba1\u7b97\u76f8\u4f3c\u5ea6\u7684\u6a21\u578bstsb-TinyBERT-L-4\u4e2d\u6587\u6548\u679c\u4e0d\u786e\u5b9a)<br \/>\ncontext_recall\uff1a \u8fd9\u4e2a\u4e0d\u786e\u5b9a\u54ea\u4e2a\u573a\u666f\u7528\u2026\u2026<br \/>\nfaithfulness\uff1a \u9760\u8c31<br \/>\nanswer_relevancy\uff1a \u5927\u6982\u7387\u9760\u8c31\uff08\u53d6\u51b3\u4e8e<code>text-embedding-ada-002<\/code>\u5411\u91cf\u5316\u540e\u8bed\u4e49\u7684\u7cbe\u51c6\u6027\uff09<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Ragas\u662f\u4e00\u4e2a\u4e13\u95e8\u7528\u4e8e\u8bc4\u4f30\u548c\u4f18\u5316\u68c0\u7d22\u589e\u5f3a\u751f\u6210\uff08RAG\uff09\u7cfb\u7edf\u7684\u5de5\u5177\u3002\u5b83\u901a\u8fc7\u5206\u6790\u67e5\u8be2\u3001\u68c0\u7d22\u4e0a\u4e0b\u6587\u548c\u751f\u6210\u7b54\u6848\u4e4b\u95f4\u7684\u5173\u7cfb\uff0c\u63d0\u4f9b\u4e86\u4e00\u5957\u5168\u9762\u7684\u8bc4\u4f30\u6307\u6807\u3002\u8fd9\u4e9b\u6307\u6807\u5305\u62ec\u5fe0\u5b9e\u5ea6\u3001\u7b54\u6848\u76f8\u5173\u6027\u3001\u4e0a\u4e0b\u6587\u76f8\u5173\u6027\u3001\u4e0a\u4e0b\u6587\u53ec\u56de\u7387\u548c\u4e0a\u4e0b\u6587\u7cbe\u786e\u5ea6\u3002Ragas\u5e2e\u52a9\u5f00\u53d1\u8005\u8bc4\u4f30&#8230;<\/p>\n","protected":false},"author":1,"featured_media":61721,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[425,20,459],"tags":[230,243],"class_list":["post-19357","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-professional","category-tool","category-rag-project","tag-aikaiyuanxiangmu","tag-aizhishikuyukefu"],"_links":{"self":[{"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/posts\/19357","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/comments?post=19357"}],"version-history":[{"count":0,"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/posts\/19357\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/media\/61721"}],"wp:attachment":[{"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/media?parent=19357"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/categories?post=19357"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kdjingpai.com\/pt\/wp-json\/wp\/v2\/tags?post=19357"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}