Chat with a local ollama model

To use chat_ollama() first download and install ollama. Then install some models from the command line, e.g. with ollama pull llama3.1 or ollama pull gemma2.

This function is a lightweight wrapper around chat_openai() with the defaults tweaked for ollama.

Usage

chat_ollama(
  system_prompt = NULL,
  turns = NULL,
  base_url = "http://localhost:11434/v1",
  model,
  seed = NULL,
  api_args = list(),
  echo = NULL
)

Arguments

system_prompt

A system prompt to set the behavior of the assistant.

turns

A list of turns to start the chat with (i.e., continuing a previous conversation). If not provided, the conversation begins from scratch. Do not provide non-NULL values for both turns and system_prompt.

Each message in the list should be a named list with at least role (usually system, user, or assistant, but tool is also possible). Normally there is also a content field, which is a string.

base_url

The base URL to the endpoint; the default uses OpenAI.

model

The model to use for the chat. The default, NULL, will pick a reasonable default, and tell you about. We strongly recommend explicitly choosing a model for all but the most casual use.

seed

Optional integer seed that ChatGPT uses to try and make output more reproducible.

api_args

Named list of arbitrary extra arguments appended to the body of every chat API call.

echo

One of the following options:

none: don't emit any output (default when running in a function).
text: echo text output as it streams in (default when running at the console).
all: echo all input and output.

Note this only affects the chat() method.

Usage

Arguments

See also