PluginsPlugin AiRag

Search​Search

Search an embedding store

Perform a semantic search using a query string.

yaml
type: "io.kestra.plugin.ai.rag.Search"

Search an embedding store

yaml
id: search_embeddings_flow
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ kv('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    drop: true
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-22.md

  - id: search
    type: io.kestra.plugin.ai.rag.Search
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ kv('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    query: "Feature Highlights"
    maxResults: 5
    minScore: 0.5
    fetchType: FETCH
Properties

Embedding store provider

Maximum number of results

Minimum similarity score

Embedding model provider

Query string

Default NONE
Possible Values
STOREFETCHFETCH_ONENONE
SubType string

List of matching text results

Count of fetched or stored items

Format uri

Output file URI in Kestra’s internal storage

Only available when fetchType is set to STORE

Unit records

The number of rows fetch from the embedding store.

The database name

The database server host

The database password

The database server port

The table to store embeddings in

The database user

Default false

Whether to use use an IVFFlat index

An IVFFlat index divides vectors into lists, and then searches a subset of those lists closest to the query vector. It has faster build times and uses less memory than HNSW but has lower query performance (in terms of speed-recall tradeoff).

Possible Values
SingleWordMaxWordMinWordSplitFuzzy
SubType string
Possible Values
LONGDOUBLEBOOLEANKEYWORDTEXTNESTEDGEO_POINTDATEVECTORFUZZY_KEYWORDIPJSONUNKNOWN
Possible Values
DOCSFREQSPOSITIONSOFFSETS
Possible Values
FLATTENNESTED
SubType string
SubType

The collection name

The host

The index name

The scheme (e.g., mongodb+srv)

Create the index

The database

SubType string

The metadata field names

The connection string options

The password

The username

API Key

Model name

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Whether to create the table if it doesn't exist

Database URL of the MariaDB database (e.g., jdbc: mariadb://host: port/dbname)

Name of the column used as the unique ID in the database

The password

Name of the table where embeddings will be stored

The username

SubType string

Metadata Column Definitions

List of SQL column definitions for metadata fields (e.g., 'text TEXT', 'source TEXT'). Required only when using COLUMN_PER_KEY storage mode.

SubType string

Metadata Index Definitions

List of SQL index definitions for metadata columns (e.g., 'INDEX idx_text (text)'). Used only with COLUMN_PER_KEY storage mode.

Metadata Storage Mode

Determines how metadata is stored: - COLUMN_PER_KEY: Use individual columns for each metadata field (requires columnDefinitions and indexes). - COMBINED_JSON (default): Store metadata as a JSON object in a single column. If columnDefinitions and indexes are provided, COLUMN_PER_KEY must be used.

The database base URL

The collection name

API Key

Model name

Default https://open.bigmodel.cn/

API base URL

The base URL for ZhiPu API (defaults to https://open.bigmodel.cn/)

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

The maximum retry times to request

The maximum number of tokens returned by this request

SubType string

With the stop parameter, the model will automatically stop generating text when it is about to contain the specified string or token_id

The database server host

The database server port

Default embedding-index

The index name

Basic authorization password

Basic authorization username

API Key

Model name

Default https://api.deepseek.com/v1

API base URL

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

The API key

The cloud provider

The index

The cloud provider region

The namespace (default will be used if not provided)

API Key

Model name

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Model endpoint

Model name

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

API Key

Model name

Default https://api.openai.com/v1

API base URL

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Possible Values
EUCLIDEANCOSINEDOT_PRODUCT

The name of the index to store embeddings

Account Identifier

Unique identifier assigned to an account

API Key

Model name

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

API endpoint

The Azure OpenAI endpoint in the format: https://{resource}.openai.azure.com/

Model name

API Key

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client ID

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Client secret

API version

Tenant ID

The API key

The collection name

The database server host

The database server port

Endpoint URL

Project location

Model name

Project ID

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Access Key ID

The access key ID used for authentication with the database.

Access Key Secret

The access key secret used for authentication with the database.

Endpoint URL

The base URL for the Tablestore database endpoint.

Instance Name

The name of the Tablestore database instance.

SubType

Metadata Schema List

Optional list of metadata field schemas for the collection.

API Key

Model name

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Default {{flow.id}}-embedding-store

The name of the KV pair to use

OCID of OCI Compartment with the model

Model name

OCI Region to connect the client to

OCI SDK Authentication provider

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Token

Milvus auth token. Required if authentication is enabled; omit for local deployments without auth.

Auto flush on delete

If true, flush after delete operations.

Auto flush on insert

If true, flush after insert operations. Setting it to false can improve throughput.

Collection name

Target collection. Created automatically if it does not exist. Default: "default".

Consistency level

Read/write consistency level. Common values include STRONG, BOUNDED, or EVENTUALLY (depends on client/version).

Database name

Logical database to use. If not provided, the default database is used.

Host

Milvus host name (used when uri is not set). Default: "localhost".

ID field name

Field name for document IDs. Default depends on collection schema.

Index type

Vector index type (e.g., IVF_FLAT, IVF_SQ8, HNSW). Depends on Milvus deployment and dataset.

Metadata field name

Field name for metadata. Default depends on collection schema.

Metric type

Similarity metric (e.g., L2, IP, COSINE). Should match the embedding provider’s expected metric.

Password

Required when authentication/TLS is enabled. See https://milvus.io/docs/authenticate.md

Port

Milvus port (used when uri is not set). Typical: 19530 (gRPC) or 9091 (HTTP). Default: 19530.

Retrieve embeddings on search

If true, return stored embeddings along with matches. Default: false.

Text field name

Field name for original text. Default depends on collection schema.

URI

Connection URI. Use either uri OR host/port (not both). Examples:

  • gRPC (typical): "milvus://host: 19530"
  • HTTP: "http://host: 9091"

Username

Required when authentication/TLS is enabled. See https://milvus.io/docs/authenticate.md

Vector field name

Field name for the embedding vector. Must match the index definition and embedding dimensionality.

API Key

Model name

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Maximum Tokens

Specifies the maximum number of tokens that the model is allowed to generate in its response.

API key

Weaviate API key. Omit for local deployments without auth.

Host

Cluster host name without protocol, e.g., "abc123.weaviate.network".

Avoid duplicates

If true (default), a hash-based ID is derived from each text segment to prevent duplicates. If false, a random ID is used.

Possible Values
ONEQUORUMALL

Consistency level

Write consistency: ONE, QUORUM (default), or ALL.

gRPC port

Port for gRPC if enabled (e.g., 50051).

Metadata field name

Field used to store metadata. Defaults to "_metadata" if not set.

SubType string

Metadata keys

The list of metadata keys to store - if not provided, it will default to an empty list.

Object class

Weaviate class to store objects in (must start with an uppercase letter). Defaults to "Default" if not set.

Port

Optional port (e.g., 443 for https, 80 for http). Leave unset to use provider defaults.

Scheme

Cluster scheme: "https" (recommended) or "http".

Secure gRPC

Whether the gRPC connection is secured (TLS).

Use gRPC for batch inserts

If true, use gRPC for batch inserts. HTTP remains required for search operations.

SubType string
Min items 1

List of HTTP Elasticsearch servers

Must be a URI like https://example.com: 9200 with scheme and port

Basic authorization configuration

SubType string

List of HTTP headers to be sent with every request

Each item is a key: value string, e.g., Authorization: Token XYZ

Path prefix for all HTTP requests

If set to /my/path, each client request becomes /my/path/ + endpoint. Useful when Elasticsearch is behind a proxy providing a base path; do not use otherwise.

Treat responses with deprecation warnings as failures

Trust all SSL CA certificates

Use this if the server uses a self-signed SSL certificate

API Key

Model name

Default https://dashscope-intl.aliyuncs.com/api/v1

API base URL

text
If you use a model in the China (Beijing) region, you need to replace the URL with: https://dashscope.aliyuncs.com/api/v1,
otherwise use the Singapore region of: "https://dashscope-intl.aliyuncs.com/api/v1.
The default value is computed based on the system timezone.

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Whether the model uses Internet search results for reference when generating text or not

The maximum number of tokens returned by this request

Repetition in a continuous sequence during model generation

text
Increasing repetition_penalty reduces the repetition in model generation,
1.0 means no penalty. Value range: (0, +inf)

API base URL

Model name

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

AWS Access Key ID

Model name

AWS Secret Access Key

Base URL

Custom base URL to override the default endpoint (useful for local tests, WireMock, or enterprise gateways).

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.

Default COHERE
Possible Values
COHERETITAN

Amazon Bedrock Embedding Model Type

API Key

Model name

Default https://router.huggingface.co/v1

API base URL

CA PEM certificate content

CA certificate as text, used to verify SSL/TLS connections when using custom endpoints.

Client PEM certificate content

PEM client certificate as text, used to authenticate the connection to enterprise AI endpoints.