API Details

API allows you to interact with Large Language Models (LLMs) and build your own AI Agents or Code Assistants. This API supports model selection, temperature control, and other options to fine-tune the output.

Request Body Fields

query

• Description: The user's input query.
• Type: string
• Required: Yes
• Example: "What is the capital of France?"

model

• Description: The LLM model to use for processing the query. The available models depend on your subscription plan. Currently, we support:
  • llama31_8b_awq
  • llama31_70b_awq
• Type: string
• Required: Yes
• Example: "llama31_70b_awq"

session_reset

• Description: Controls whether to reset the session and generate a new chat_id for further discussion.
• Type: boolean
• Default: false
• Example: true or false

user_store_ids

• Description: A comma-separated list of IDs from your storage records that provide access to RAG context.
• Type: string
• Required: No
• Example: "1,2,3"

model_options

• Description: Options for the selected model. See below for details.
• Type: object

Model Options

The model_options field allows you to control various aspects of the LLM output.

temperature

• Description: The temperature value controls the randomness of the output. Higher values result in more random responses.
• Type: number
• Default (Llama 3.1): 1.0
• Example: 0.5 or 2.0

max_tokens

• Description: The maximum number of tokens to generate in the response.
• Type: integer
• Default (Llama 3.1): 200
• Example: 100 or 500

top_k and top_p

• Description: The top-k value controls the number of highest-probability words to consider when generating text, while top-p controls the cumulative probability threshold for considering words.
• Type: integer (top_k) or number (top_p)
• Default (Llama 3.1): not set
• Example: 5 or 10 (top_k), 0.9 or 0.95 (top_p)

Store Options

The store_options field allows you to control RAG settings.

limited_by_context

• Description: Controls whether the AI should reply with "I don't know" if context is not provided.
• Type: boolean
• Default: true
• Example: true or false

Example Request Body

{
  "query": "What is the capital of France?",
  "model": "llama31_70b_awq",
  "session_reset": false,
  "user_store_ids": "1,2,3",
  "model_options": {
    "temperature": 1.0,
    "max_tokens": 200
  },
  "store_options": {
    "limited_by_context": true
  }
}

Note that some fields have default values, which are used if not explicitly provided in the request body.