haive.core.models.llm.providers.replicate

Replicate Provider Module.

This module implements the Replicate language model provider for the Haive framework, supporting a wide variety of open-source models hosted on Replicate’s platform.

The provider handles API key management, model configuration, and safe imports of the langchain-community package dependencies for Replicate integration.

Examples

Basic usage:

from haive.core.models.llm.providers.replicate import ReplicateProvider

provider = ReplicateProvider(
    model="meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3",
    temperature=0.7,
    max_tokens=1000
)
llm = provider.instantiate()

With custom parameters:

provider = ReplicateProvider(
    model="mistralai/mixtral-8x7b-instruct-v0.1",
    temperature=0.1,
    top_p=0.9,
    top_k=50
)

ReplicateProvider(*[, requests_per_second, ...])

Replicate language model provider configuration.

Classes

ReplicateProvider

Replicate language model provider configuration.

Module Contents

class haive.core.models.llm.providers.replicate.ReplicateProvider(/, **data)

Bases: haive.core.models.llm.providers.base.BaseLLMProvider

Replicate language model provider configuration.

This provider supports a wide variety of open-source models hosted on Replicate, including Llama, Mixtral, CodeLlama, and many others with flexible versioning.

Parameters:

• data (Any)

• requests_per_second (float | None)

• tokens_per_second (int | None)

• tokens_per_minute (int | None)

• max_retries (int)

• retry_delay (float)

• check_every_n_seconds (float | None)

• burst_size (int | None)

• provider (LLMProvider)

• model (str | None)

• name (str | None)

• api_key (SecretStr)

• cache_enabled (bool)

• cache_ttl (int | None)

• extra_params (dict[str, Any] | None)

• debug (bool)

• temperature (float | None)

• max_tokens (int | None)

• top_p (float | None)

• top_k (int | None)

• repetition_penalty (float | None)

• stop_sequences (list[str] | None)

provider

Always LLMProvider.REPLICATE

Type:

LLMProvider

model

The Replicate model to use (owner/name:version format)

Type:

str

temperature

Sampling temperature (0.0-5.0)

Type:

float

max_tokens

Maximum tokens in response

Type:

int

top_p

Nucleus sampling parameter

Type:

float

top_k

Top-k sampling parameter

Type:

int

repetition_penalty

Repetition penalty parameter

Type:

float

stop_sequences

Stop sequences for generation

Type:

list

Examples

Llama 2 70B model:

provider = ReplicateProvider(
    model="meta/llama-2-70b-chat",
    temperature=0.7,
    max_tokens=2000
)

Mixtral with specific version:

provider = ReplicateProvider(
    model="mistralai/mixtral-8x7b-instruct-v0.1:7b3212fbaf88310047672c7764d9f2cce7493d0d80666d899b72af8c0662df7a",
    temperature=0.1,
    top_p=0.9
)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

classmethod get_models()

Get popular Replicate models.

Return type:

list[str]

classmethod validate_model_format(v)

Validate Replicate model format.

Parameters:

v (str)

Return type:

str

max_tokens: int | None = None

Get maximum total tokens for this model.