LLMService gRPC API
LLMService provides provider-agnostic LLM access for plugins. Provider plugins (OpenRouter, llama.cpp) register as backends. Core routes requests to the appropriate provider.
Proto file: plugin/grpc/protocol/llm.proto
Service Methods
| Method | Request | Response | Streaming |
|---|
| Chat | LLMChatRequest | LLMChatResponse | No |
| StreamChat | LLMChatRequest | LLMChatChunk | Server |
Chat
- Request:
LLMChatRequest
- Response:
LLMChatResponse
StreamChat
- Request:
LLMChatRequest
- Response:
LLMChatChunk
- Streaming: server
Message Types
ChatMessage
| Field | Type | Description |
|---|
| role | string | - |
| content | string | - |
LLMChatRequest
| Field | Type | Description |
|---|
| system_prompt | string | Deprecated: use messages field for new callers. |
| user_prompt | string | Deprecated: use messages field for new callers. |
| model | string | - |
| temperature | double | - |
| max_tokens | int32 | - |
| provider | string | - |
| attachments | Attachment | - |
| messages | ChatMessage | Multi-turn conversation history. When populated, takes precedence over system_prompt/user_prompt which remain for single-turn callers. |
| priority | int32 | Queuing priority. Lower value = higher priority. 0 = interactive (user-initiated), 10 = background (generators, batch). |
Attachment
| Field | Type | Description |
|---|
| mime_type | string | - |
| data | string | - |
| filename | string | - |
LLMChatResponse
| Field | Type | Description |
|---|
| content | string | - |
| model | string | - |
| prompt_tokens | int32 | - |
| completion_tokens | int32 | - |
| total_tokens | int32 | - |
LLMChatChunk
Streaming: one chunk per generated token
| Field | Type | Description |
|---|
| token | string | - |
| done | bool | - |
| model | string | - |
| prompt_tokens | int32 | - |
| completion_tokens | int32 | - |
| total_tokens | int32 | - |
| signal | TokenSignalProto | - |
TokenSignalProto
| Field | Type | Description |
|---|
| confidence | float | - |
| entropy | float | - |
| top_gap | float | - |
| top_k | TokenCandidateProto | - |
| full_distribution | float | - |
| sampler_stages | SamplerStageProto | - |
TokenCandidateProto
| Field | Type | Description |
|---|
| id | int32 | - |
| text | string | - |
| prob | float | - |
SamplerStageProto
Snapshot of token distribution after a sampler stage
| Field | Type | Description |
|---|
| name | string | - |
| active_count | int32 | - |
| top1_prob | float | - |
| entropy | float | - |
| top_k | TokenCandidateProto | - |
← Back to API Index