참고소스 수정본

참고/instructor-main/docs/prompting/ensembling/more.md

@@ -0,0 +1,161 @@
+---
+description: "MoRE creates a set of diverse reasoning experts by using different specialized prompts for different reasoning types. THe best answer from all experts is then selected using an agreement score"
+---
+
+Language Models struggle to generalize across question types that require distinct reasoning abilities. By combining a variety of different specialized language models, we can improve the quality of our responses. This is done through a technique called Mixture Of Reasoning Experts (MoRE).
+
+In the original paper, they utilise four different experts
+
+1. Factual Expert : This is a model that is augmented by a RAG prompting pipeline. WHen it recieves a query, it retrieves the top 10 most relevant passages from Wikipedia and appends them to the prompt right before the question.
+
+2. Multihop Expert : This is an expert that has manually written rationales after each demo to elicit multi-step reasoning processes for the questions
+
+3. Math Expert : This is an expert that has manually written explanations for the GSM8k Dataset to bias the model towards different reasoning steps
+
+4. Commonsense expert: This is an expert that is provided with 10 different facts that are generated by a Codex model which are appended to the prompt right before the question
+
+![](../../img/more.png)
+
+Once each expert has genearted a response, they then use a random forest classifier to score it from 0 to 1. This is then used for selecting the final answer and determining if we've generated a sufficiently good answer ( Since we have the option to abstain at each point )
+
+We can implement a simplified version of MoRE with `instructor` with a few modifications.
+
+```python
+from pydantic import BaseModel, Field
+import instructor
+from textwrap import dedent
+client = instructor.from_provider("openai/gpt-5-nano")
+
+
+class MultihopExpert(BaseModel):
+    chain_of_thought: str
+    answer: str
+
+
+class FactualExpert(BaseModel):
+    answer: str
+
+
+class ModelScore(BaseModel):
+    score: float = Field(ge=0, lt=1)
+
+
+def query_factual_expert(query: str, evidence: list[str]):
+    formatted_evidence = "\n-".join(evidence)
+    return client.create(
+        model="gpt-4o",
+        response_model=FactualExpert,
+        messages=[
+            {
+                "role": "system",
+                "content": dedent(
+                    f"""
+                <query>
+                {query}
+                </query>
+
+                <evidences>
+                {formatted_evidence}
+                </evidences>
+                """
+                ),
+            }
+        ],
+    )
+
+
+def query_multihop_expert(query: str):
+    return client.create(
+        model="gpt-4o",
+        response_model=MultihopExpert,
+        messages=[
+            {
+                "role": "system",
+                "content": dedent(
+                    f"""
+                <query>
+                {query}
+                </query>
+                """
+                ),
+            }
+        ],
+    )
+
+
+def score_answer(query: str, answer: str):
+    return client.create(
+        model="gpt-4o",
+        response_model=ModelScore,
+        messages=[
+            {
+                "role": "system",
+                "content": """You are a helpful assistant that scores
+                answers based on well they are able to answer a
+                specific user query""",
+            },
+            {
+                "role": "user",
+                "content": f"""
+                <user query>
+                {query}
+                </user query>
+
+                <response>
+                {answer}
+                </response>
+                """,
+            },
+        ],
+    )
+
+
+if __name__ == "__main__":
+    query = """Who's the original singer of Help Me Make It
+    Through The Night?"""
+    evidences = [
+        """Help Me Make It Through The Night is a country
+        music ballad written and composed by Kris Kristofferson
+        and released on his 1970 album 'Kristofferson'"""
+    ]
+
+    threshold = 0.8
+
+    factual_expert_output = query_factual_expert(query, evidences)
+    print(factual_expert_output.model_dump_json(indent=2))
+    """
+    {
+      "answer": "The original singer of 'Help Me Make It Through the
+      Night' is Kris Kristofferson, who released it on his 1970 album
+      'Kristofferson'."
+    }
+    """
+
+    multihop_expert_output = query_multihop_expert(query)
+    print(multihop_expert_output.model_dump_json(indent=2))
+    """
+    {
+      "chain_of_thought": "To identify the original singer of 'Help Me
+      Make It Through The Night,' I need to look for the person who
+      first recorded and released the song.",
+      "answer": "The original singer of 'Help Me Make It Through
+      The Night' is Kris Kristofferson."
+    }
+    """
+
+    factual_expert_score = score_answer(query, factual_expert_output.answer)
+    multihop_expert_score = score_answer(query, multihop_expert_output.answer)
+
+    if max(factual_expert_score.score, multihop_expert_score.score) < threshold:
+        answer = "Abstaining from responding"
+    elif factual_expert_score.score > multihop_expert_score.score:
+        answer = factual_expert_output.answer
+    else:
+        answer = multihop_expert_output.answer
+
+    print(answer)
+    """
+    The original singer of 'Help Me Make It Through the Night' is Kris
+    Kristofferson, who released it on his 1970 album 'Kristofferson'.
+    """
+```