# Reranking Functions

NOTE

This feature is available starting from version 1.2.1.

The Rerank function in MyScale enhances the relevance of search results by reordering them based on their semantic similarity to a query, further improving retrieval accuracy and reducing cost in retrieval-augmented generation (RAG) by removing undesired results. It supports providers like HuggingFace, Amazon SageMaker, Cohere and Jina AI offering flexibility in choosing the best-suited provider for specific needs.

Syntax

Rerank(query, documents, top_k, provider, base_url, api_key, others)

Arguments

query (String): The search query, non-empty string.
documents (Array(String)): A list of document strings to rerank.
top_k (UInt32): The number of most relevant documents to return.
provider (String): Must be one of the following rerank model providers, case-insensitive - HuggingFace, SageMaker, Cohere, Jina.
base_url (String): Provider's Rerank API URL, optional for some.
api_key (String): Non-empty Rerank API key for authentication.
others (String): Additional provider-specific parameters that should be provided as a JSON map. Details are provide in their respective descriptions.

Returned Value

The returned value from the Rerank function is an array of tuples, each tuple consisting of three elements: the index, the document text, and the score. This array is sorted in descending order based on the score, ensuring that the documents most relevant to the query are listed first. Each tuple's index corresponds to the original position of the document in the input array, allowing for easy reference.
Type Array(Tuple(UInt32, String, Float32)).

# HuggingFace Rerank

Setting the provider parameter to HuggingFace in Rerank uses the HuggingFace Inference API (opens new window) or Inference Endpoint (opens new window) for reranking.

Note: This feature is specifically designed for APIs with particular input and output formats. The expected input format is a JSON object with an "inputs" array, where each element contains a "text" (query) and "text_pair" (doc in documents). The output from the API is anticipated to be a list of objects, each with a "score" key and possibly other data. These outputs are arranged in the order corresponding to their respective pairs in the "inputs" array, where each output score is associated with the respective input element.

For instance, a sample input could be:

{
  "inputs": [
    {"text": "what is a panda?", "text_pair": "hi."},
    {"text": "what is a panda?", "text_pair": "Giant panda is characterized by its bold black-and-white coat and rotund body."}
  ]
}

The corresponding output would be:

[
    [
        {
            "score": 5.5232932936633006e-05
        }
    ],
    [
        {
            "score": 0.6181288361549377
        }
    ]
]

Models like BAAI/bge-reranker-base (opens new window) and amberoad/bert-multilingual-passage-reranking-msmarco (opens new window) are compatible.

Provider-specific parameters

base_url: HuggingFace Rerank API URL. Required.
api_key: HuggingFace API Key. Required.
others :
- model_args: Optional parameters specific to the HuggingFace model being used.

Examples

Using Default Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'HuggingFace', 'HUGGINGFACE_ENDPOINT', 'HUGGINGFACE_API_KEY', '')

Using Custom Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'HuggingFace', 'HUGGINGFACE_ENDPOINT', 'HUGGINGFACE_API_KEY', '{"model_args": {"parameters":{"truncation":true}}}')

Simplified usage with custom function:

CREATE FUNCTION HuggingFaceRerank ON CLUSTER '{cluster}' AS (x,y,z) -> Rerank(x, y, z, 'HuggingFace', 'HUGGINGFACE_ENDPOINT', 'HUGGINGFACE_API_KEY', '')
SELECT HuggingFaceRerank('Query', ['doc', ...], TOP_K)

# Amazon SageMaker Rerank

Setting the provider parameter to SageMaker in Rerank uses the Amazon SageMaker Endpoints (opens new window) for reranking.

Note: This provider is specifically designed for models deployed on Amazon SageMaker, requiring input and output formats identical to those used by the HuggingFace rerank models.

You can find models that comply with these specifications on HuggingFace, such as BAAI/bge-reranker-base (opens new window). Additionally, you can access SageMaker deployment code as illustrated in the picture below:

sagemaker-deploy

Provider-specific parameters

base_url: SageMaker Endpoint name. Required.
api_key: AWS secret_access_key. Required.
others:
- access_key_id: AWS access_key_id. Required.
- region_name: AWS region name. Required.
- model_args: Optional parameters specific to the SageMaker endpoint being used.

Examples

Using Default Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'SageMaker', 'SAGEMAKER_ENDPOINT', 'AWS_ACCESS_KEY', '{"region_name":"us-east-1", "access_key_id":"ACCESS_KEY_ID"}')

Using Custom Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'SageMaker', 'SAGEMAKER_ENDPOINT', 'SAGEMAKER_API_KEY', '{"region_name":"us-east-1", "access_key_id":"ACCESS_KEY_ID", "model_args": {"parameters":{"truncation":true}}}')

Simplified usage with custom function:

CREATE FUNCTION SageMakerRerank ON CLUSTER '{cluster}' AS (x,y,z) -> Rerank(x, y, z, 'SageMaker', 'SAGEMAKER_ENDPOINT', 'SAGEMAKER_API_KEY', '{"region_name":"us-east-1", "access_key_id":"ACCESS_KEY_ID"}')
SELECT SageMakerRerank('Query', ['doc', ...], TOP_K)

# Cohere Rerank

Setting the provider parameter to Cohere in Rerank uses the Cohere Rerank API (opens new window) for reranking.

Provider-specific parameters

base_url: Cohere Rerank API URL. Optional. Default value is https://api.cohere.ai/v1/rerank (opens new window)
api_key: Cohere API Key. Required.
others:
- model: Model ID to use. Optional. Default value is rerank-english-v2.0
- max_chunks_per_doc: The maximum number of chunks to produce internally from a document. Optional.

Examples

Using Default Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'Cohere', '', 'COHERE_API_KEY', '')

Using Custom Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'Cohere', 'YOUR_RERANK_API_URL', 'COHERE_API_KEY', '{"model":"MODEL_ID", "max_chunks_per_doc":CHUNK_NUMBER}')

Simplified usage with custom function:

CREATE FUNCTION CohereRerank ON CLUSTER '{cluster}' AS (x,y,z) -> Rerank(x, y, z, 'Cohere', 'YOUR_RERANK_API_URL', 'COHERE_API_KEY', '')
SELECT CohereRerank('Query', ['doc', ...], TOP_K)

# Jina AI Rerank

Setting the provider parameter to Jina in Rerank uses the Jina AI Rerank API (opens new window) for reranking.

Provider-specific parameters

base_url: Jina AI Rerank API URL. Optional. Default value is https://api.jina.ai/v1/rerank (opens new window)
api_key: Jina AI API Key. Required.
others:
- model: Model ID to use. Optional. Default value is jina-reranker-v1-base-en

Examples

Using Default Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'Jina', '', 'JINAAI_API_KEY', '')

Using Custom Values:

SELECT Rerank('Query', ['doc', ...], TOP_K, 'Jina', 'YOUR_RERANK_API_URL', 'JINAAI_API_KEY', '{"model":"MODEL_ID"}')

Simplified usage with custom function:

CREATE FUNCTION JinaAIRerank ON CLUSTER '{cluster}' AS (x,y,z) -> Rerank(x, y, z, 'Jina', 'YOUR_RERANK_API_URL', 'JINAAI_API_KEY', '')
SELECT JinaAIRerank('Query', ['doc', ...], TOP_K)