Prompt Generator

rankify.generator.prompt_generator

DEFAULT_PROMPTS = {PromptTemplate.BASIC_RAG: 'You are a helpful assistant. Give a single, concise answer to the question using the provided contexts.\nIf the provided contexts are not sufficient, you may also use your own knowledgeQuestion: {question}\nContexts:\n{contexts}\nAnswer:', PromptTemplate.CHAIN_OF_THOUGHT_RAG: "You are a helpful assistant. Think step by step and then answer concisely with only the answer as a fact.\nUse the provided contexts if possible; if they are insuffiecient, you may use your own knowledge.Question: {question}\nContexts:\n{contexts}\nLet's think step by step.\nAnswer:", PromptTemplate.SELF_CONSISTENCY_RAG: "You are a helpful assistant. Think step by step and then answer concisely with only the answer as a fact.\nUse the provided contexts if possible; if they are insuffiecient, you may use your own knowledge.Question: {question}\nContexts:\n{contexts}\nLet's think step by step.\nAnswer:", PromptTemplate.ZERO_SHOT: 'Answer the following question concisely.\nQuestion: {question}\nAnswer:', PromptTemplate.REACT_RAG: 'You are an intelligent assistant that solves tasks by interleaving reasoning with actions.\n\nYou must follow this format strictly:\nThought: [your reasoning about the problem]\nAction: [the action you want to take, in the form Search[argument]]\nObservation: [the result returned by the environment]\n\nRepeat Thought → Action → Observation as many times as needed.\nIf you are confident you know the answer from your own knowledge, you may proceed directly to the Final Answer without searching.\nWhen you have enough information, provide:\nFinal Answer: [your final solution to the user’s query]\n\nGuidelines:\n- Be concise in each Thought, just explain what is necessary for your next step.\n- Only use the defined Actions available to you (Search[query]).\n- If no action is required or you are sure of the answer, proceed to the Final Answer.\n- Do not hallucinate Observations; they are always provided externally.\n- End with "Final Answer:" when the task is solved.\n\n- The final Answer should be as precise and concise as possible.\nQuestion: {question}\nContexts:\n{contexts}\nThought:', PromptTemplate.IN_CONTEXT_RALM: 'You are a helpful assistant. Give a single, concise answer to the question using the provided contexts.\nIf the provided contexts are not sufficient, you may also use your own knowledgeQuestion: {question}\nContexts:\n{contexts}\nAnswer:'} module-attribute

PromptTemplate

Bases: Enum

PromptTemplate Enum for RAG Prompt Strategies.

Enumerates the supported prompt templates for different RAG methods in Rankify. Each value corresponds to a specific prompt formatting strategy, used by the PromptGenerator to construct prompts for the model.

Members

BASIC_RAG: Standard prompt for naive RAG (context + question + answer). CHAIN_OF_THOUGHT_RAG: Prompt for chain-of-thought reasoning (step-by-step thinking). ZERO_SHOT: Prompt for zero-shot generation (question only, no context). SELF_CONSISTENCY_RAG: Prompt for self-consistency reasoning (multiple step-by-step answers). REACT_RAG: Prompt for ReAct reasoning and action (interleaved reasoning and retrieval actions).

Notes

• Used by PromptGenerator to select and format prompts for each RAG method.
• Ensures consistent prompt structure across different techniques and models.
• The corresponding template strings are defined in DEFAULT_PROMPTS.

Source code in rankify/generator/prompt_template.py

class PromptTemplate(Enum):
    """
    **PromptTemplate Enum** for RAG Prompt Strategies.

    Enumerates the supported prompt templates for different RAG methods in Rankify.
    Each value corresponds to a specific prompt formatting strategy, used by the PromptGenerator to construct prompts for the model.

    Members:
        BASIC_RAG: Standard prompt for naive RAG (context + question + answer).
        CHAIN_OF_THOUGHT_RAG: Prompt for chain-of-thought reasoning (step-by-step thinking).
        ZERO_SHOT: Prompt for zero-shot generation (question only, no context).
        SELF_CONSISTENCY_RAG: Prompt for self-consistency reasoning (multiple step-by-step answers).
        REACT_RAG: Prompt for ReAct reasoning and action (interleaved reasoning and retrieval actions).

    Notes:
        - Used by PromptGenerator to select and format prompts for each RAG method.
        - Ensures consistent prompt structure across different techniques and models.
        - The corresponding template strings are defined in DEFAULT_PROMPTS.
    """

    BASIC_RAG = "basic-rag"
    CHAIN_OF_THOUGHT_RAG = "chain-of-thought-rag"
    ZERO_SHOT = "zero-shot"
    SELF_CONSISTENCY_RAG = "self-consistency-rag"
    REACT_RAG = "react-rag"
    IN_CONTEXT_RALM="in-context-ralm"

PromptGenerator

PromptGenerator for Retrieval-Augmented Generation (RAG).

This class manages prompt construction for different RAG methods and model types in Rankify. It selects and formats prompt templates based on the specified method and model, enabling flexible and consistent prompt generation.

Attributes:

Name	Type	Description
method	str	The RAG method or strategy (e.g., "basic-rag", "chain-of-thought-rag").
prompt_template	PromptTemplate	The prompt template used for formatting prompts.
model_type	str	The type of model (e.g., "huggingface", "openai").

Methods:

Name	Description
generate_user_prompt	Generates a user prompt by formatting the question and contexts with the selected template.
_select_template	Selects the appropriate prompt template for the given method.

Notes

• Supports custom prompt templates via the custom_prompt argument.
• If no contexts are provided, generates prompts with only the question.
• Ensures consistent prompt formatting across different RAG methods and models.
• Automatically selects a default template if none is specified.

Source code in rankify/generator/prompt_generator.py

class PromptGenerator:
    """
    **PromptGenerator** for Retrieval-Augmented Generation (RAG).

    This class manages prompt construction for different RAG methods and model types in Rankify.
    It selects and formats prompt templates based on the specified method and model, enabling flexible and consistent prompt generation.

    Attributes:
        method (str): The RAG method or strategy (e.g., "basic-rag", "chain-of-thought-rag").
        prompt_template (PromptTemplate): The prompt template used for formatting prompts.
        model_type (str): The type of model (e.g., "huggingface", "openai").

    Methods:
        generate_user_prompt(question, contexts, custom_prompt=None) -> str:
            Generates a user prompt by formatting the question and contexts with the selected template.
        _select_template(method) -> PromptTemplate:
            Selects the appropriate prompt template for the given method.

    Notes:
        - Supports custom prompt templates via the `custom_prompt` argument.
        - If no contexts are provided, generates prompts with only the question.
        - Ensures consistent prompt formatting across different RAG methods and models.
        - Automatically selects a default template if none is specified.
    """
    def __init__(self, method: str, model_type: str, prompt_template: Optional[PromptTemplate] = None):
        """
        Initialize the PromptGenerator.

        Args:
            method (str): The RAG method or strategy (e.g., "basic-rag", "chain-of-thought-rag").
            model_type (str): The type of model (e.g., "huggingface", "openai").
            prompt_template (PromptTemplate, optional): Custom prompt template to use. If None, selects based on method.

        Notes:
            - If no custom template is provided, selects a default template for the specified method.
            - Stores the method, model type, and selected prompt template for prompt generation.
        """
        self.method = method
        if prompt_template is not None:
            self.prompt_template = prompt_template
        else:
            self.prompt_template = self._select_template(method)

    def _select_template(self, method: str) -> PromptTemplate:
        """
        Selects the appropriate prompt template for the given RAG method.

        Args:
            method (str): The RAG method or strategy.

        Returns:
            PromptTemplate: The selected prompt template.

        Notes:
            - If the method is not recognized, defaults to BASIC_RAG template.
        """
        method = method.lower()
        if method in PromptTemplate._value2member_map_:
            return PromptTemplate(method)
        return PromptTemplate.BASIC_RAG

    def generate_user_prompt(self, question: str, contexts: List[str], custom_prompt: Optional[str] = None) -> str:
        """
        Generates a user prompt by formatting the question and contexts with the selected template.

        Args:
            question (str): The question to be answered.
            contexts (List[str]): List of context passages to include in the prompt.
            custom_prompt (str, optional): Custom prompt template string. If provided, overrides the default template.

        Returns:
            str: The formatted prompt string.

        Notes:
            - If no contexts are provided, generates a prompt with only the question.
            - Custom prompt templates can use `{question}` and `{contexts}` placeholders.
            - Ensures consistent prompt formatting for all RAG methods and models.
        """
        if contexts is None:
            contexts = []
        context_str = "\n".join(contexts)
        if custom_prompt:
            return custom_prompt.format(question=question, contexts=context_str)
        template = DEFAULT_PROMPTS[self.prompt_template]
        return template.format(question=question, contexts=context_str)

__init__(method, model_type, prompt_template=None)

Initialize the PromptGenerator.

Parameters:

Name	Type	Description	Default
method	str	The RAG method or strategy (e.g., "basic-rag", "chain-of-thought-rag").	required
model_type	str	The type of model (e.g., "huggingface", "openai").	required
prompt_template	PromptTemplate	Custom prompt template to use. If None, selects based on method.	None

Notes

• If no custom template is provided, selects a default template for the specified method.
• Stores the method, model type, and selected prompt template for prompt generation.

Source code in rankify/generator/prompt_generator.py

def __init__(self, method: str, model_type: str, prompt_template: Optional[PromptTemplate] = None):
    """
    Initialize the PromptGenerator.

    Args:
        method (str): The RAG method or strategy (e.g., "basic-rag", "chain-of-thought-rag").
        model_type (str): The type of model (e.g., "huggingface", "openai").
        prompt_template (PromptTemplate, optional): Custom prompt template to use. If None, selects based on method.

    Notes:
        - If no custom template is provided, selects a default template for the specified method.
        - Stores the method, model type, and selected prompt template for prompt generation.
    """
    self.method = method
    if prompt_template is not None:
        self.prompt_template = prompt_template
    else:
        self.prompt_template = self._select_template(method)

generate_user_prompt(question, contexts, custom_prompt=None)

Generates a user prompt by formatting the question and contexts with the selected template.

Parameters:

Name	Type	Description	Default
question	str	The question to be answered.	required
contexts	List[str]	List of context passages to include in the prompt.	required
custom_prompt	str	Custom prompt template string. If provided, overrides the default template.	None

Returns:

Name	Type	Description
str	str	The formatted prompt string.

Notes

• If no contexts are provided, generates a prompt with only the question.
• Custom prompt templates can use {question} and {contexts} placeholders.
• Ensures consistent prompt formatting for all RAG methods and models.

Source code in rankify/generator/prompt_generator.py

def generate_user_prompt(self, question: str, contexts: List[str], custom_prompt: Optional[str] = None) -> str:
    """
    Generates a user prompt by formatting the question and contexts with the selected template.

    Args:
        question (str): The question to be answered.
        contexts (List[str]): List of context passages to include in the prompt.
        custom_prompt (str, optional): Custom prompt template string. If provided, overrides the default template.

    Returns:
        str: The formatted prompt string.

    Notes:
        - If no contexts are provided, generates a prompt with only the question.
        - Custom prompt templates can use `{question}` and `{contexts}` placeholders.
        - Ensures consistent prompt formatting for all RAG methods and models.
    """
    if contexts is None:
        contexts = []
    context_str = "\n".join(contexts)
    if custom_prompt:
        return custom_prompt.format(question=question, contexts=context_str)
    template = DEFAULT_PROMPTS[self.prompt_template]
    return template.format(question=question, contexts=context_str)