How can I help you?
Integrate LLM via Ollama With React Inline AI Assist component
19 Mar 202624 minutes to read
The Inline AI Assist component integrates with LLM via Ollama to enable advanced conversational AI features in your application. The component acts as a user interface where user prompts are sent to the selected LLM model via API calls, providing natural language understanding and context-aware responses.
Prerequisites
Before starting, ensure you have the following:
-
Node.js: Version 16 or higher with npm.
-
Ollama installed to run and manage LLM models locally.
-
Syncfusion Inline AI Assist: Package @syncfusion/ej2-react-interactive-chat installed.
-
Marked Library: For parsing Markdown responses (
npm install marked --save).
Set Up the Environment
Follow the Getting Started guide to configure and render the Inline AI Assist component in your application.
Install Dependency
To install the marked library, run npm install marked --save in your project directory to add it as a dependency in your package.json file.
Configuring Ollama
Install Ollama for your operating system:
1. Visit [Windows](https://ollama.com/download)
2. Click `Download for Windows` to get the `.exe installer`.
3. Run `OllamaSetup.exe` and follow the wizard to install.1. Visit [macOS](https://ollama.com/download/mac)
2. Click `Download for macOS` to get `.dmg file`
3. Install it by following the wizard.1. Visit [Linux](https://ollama.com/download/linux)
2. Run the below command to install Ollama in your system
```bash
curl -fsSL https://ollama.ai/install.sh | sh
```Download and run an Ollama model
- Download and run a model using the following command. Replace
deepseek-r1with your preferred model (e.g.,llama3,phi4). See the Ollama model library for available models.
ollama run deepseek-r1- After the model download completes, start the Ollama server to make the model accessible:
ollama serveConfigure Inline AI Assist with Ollama
Modify the index.tsx file to integrate the Ollama model with the Inline AI Assist component.
import { InlineAIAssistComponent } from '@syncfusion/ej2-react-interactive-chat';
import { marked } from 'marked';
import * as React from 'react';
import * as ReactDOM from 'react-dom';
function App() {
const assistRef = React.useRef(null);
const editableRef = React.useRef(null);
const [stopStreaming, setStopStreaming] = React.useState(false);
const handleSummarizeBtnClick = () => {
if (assistRef.current && typeof assistRef.current.showPopup === 'function') {
assistRef.current.showPopup();
}
};
// Stream AI response in chunks
const streamResponse = async (response) => {
let lastResponse = "";
const responseUpdateRate = 10;
let i = 0;
const responseLength = response.length;
while (i < responseLength && !stopStreaming) {
lastResponse += response[i];
i++;
if (i % responseUpdateRate === 0 || i === responseLength) {
const htmlResponse = await marked.parse(lastResponse);
if (assistRef.current && typeof assistRef.current.addResponse === 'function') {
assistRef.current.addResponse(htmlResponse, i === responseLength);
}
}
await new Promise(resolve => setTimeout(resolve, 15));
}
};
// Handle user prompt: call local LLM via Ollama
const handlePromptRequest = async (args) => {
const defaultResponse = "⚠️ Something went wrong while connecting to the AI service. Please check your Ollama application running in background.";
try {
// Send request to Ollama API
const response = await fetch('http://localhost:11434/api/generate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'deepseek-r1',
prompt: `### Instruction:\nRespond in up to 5 lines.\n\n### Input:\n${args.prompt}`,
stream: false,
}),
});
const reply = await response.json();
if (response) {
setStopStreaming(false);
await streamResponse(reply.response);
}
} catch (error) {
if (assistRef.current && typeof assistRef.current.addResponse === 'function') {
assistRef.current.addResponse(defaultResponse, true);
}
}
};
const handleResponseItemSelect = (args) => {
if (args.command.label === 'Accept') {
if (editableRef.current && assistRef.current) {
const lastResponse = assistRef.current.prompts && assistRef.current.prompts.length > 0
? assistRef.current.prompts[assistRef.current.prompts.length - 1].response
: '';
if (lastResponse) {
editableRef.current.innerHTML = '<p>' + lastResponse + '</p>';
}
}
if (assistRef.current && typeof assistRef.current.hidePopup === 'function') {
assistRef.current.hidePopup();
}
} else if (args.command.label === 'Discard') {
if (assistRef.current && typeof assistRef.current.hidePopup === 'function') {
assistRef.current.hidePopup();
}
}
};
const handleToolbarItemClick = (args) => {
if (args.item.iconCss === 'e-icons e-inline-stop') {
setStopStreaming(true);
}
};
return (
<div>
<button
id="summarizeBtn"
className="e-btn e-primary"
style=
onClick={handleSummarizeBtnClick}
>
Content Summarize
</button>
<div
id="editableText"
contentEditable="true"
ref={editableRef}
>
<p>Inline AI Assist component provides intelligent text processing capabilities that enhance user productivity. It leverages advanced natural language processing to understand context and deliver precise suggestions. Users can seamlessly integrate AI-powered features into their applications.</p>
<p>With real-time response streaming and customizable prompts, developers can create interactive experiences. The component supports multiple response modes including inline editing and popup-based interactions.</p>
</div>
<InlineAIAssistComponent
id="defaultInlineAssist"
ref={assistRef}
relateTo="#summarizeBtn"
promptRequest={handlePromptRequest}
responseSettings=
inlineToolbarSettings=
popupWidth="500px"
/>
</div>
);
}
ReactDOM.render(<App />, document.getElementById('container'));import { InlineAIAssistComponent, ResponseItemSelectEventArgs, InlinePromptRequestEventArgs, ToolbarItemClickEventArgs } from '@syncfusion/ej2-react-interactive-chat';
import { marked } from 'marked';
import * as React from 'react';
const App: React.FC = () => {
const assistRef = React.useRef<InlineAIAssistComponent>(null);
const editableRef = React.useRef<HTMLDivElement>(null);
const [stopStreaming, setStopStreaming] = React.useState(false);
const handleSummarizeBtnClick = (): void => {
if (assistRef.current && typeof assistRef.current.showPopup === 'function') {
assistRef.current.showPopup();
}
};
// Stream AI response in chunks
const streamResponse = async (response: string): Promise<void> => {
let lastResponse = "";
const responseUpdateRate = 10;
let i = 0;
const responseLength = response.length;
while (i < responseLength && !stopStreaming) {
lastResponse += response[i];
i++;
if (i % responseUpdateRate === 0 || i === responseLength) {
const htmlResponse = await marked.parse(lastResponse);
if (assistRef.current && typeof assistRef.current.addResponse === 'function') {
assistRef.current.addResponse(htmlResponse, i === responseLength);
}
}
await new Promise(resolve => setTimeout(resolve, 15));
}
};
// Handle user prompt: call local LLM via Ollama
const handlePromptRequest = async (args: InlinePromptRequestEventArgs): Promise<void> => {
const defaultResponse = "⚠️ Something went wrong while connecting to the AI service. Please check your Ollama application running in background.";
try {
// Send request to Ollama API
const response = await fetch('http://localhost:11434/api/generate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: 'deepseek-r1',
prompt: `### Instruction:\nRespond in up to 5 lines.\n\n### Input:\n${args.prompt}`,
stream: false,
}),
});
const reply = await response.json();
if (response) {
setStopStreaming(false);
await streamResponse(reply.response);
}
} catch (error) {
if (assistRef.current && typeof assistRef.current.addResponse === 'function') {
assistRef.current.addResponse(defaultResponse, true);
}
}
};
const handleResponseItemSelect = (args: ResponseItemSelectEventArgs): void => {
if (args.command.label === 'Accept') {
if (editableRef.current && assistRef.current) {
const lastResponse = assistRef.current.prompts && assistRef.current.prompts.length > 0
? assistRef.current.prompts[assistRef.current.prompts.length - 1].response
: '';
if (lastResponse) {
editableRef.current.innerHTML = '<p>' + lastResponse + '</p>';
}
}
if (assistRef.current && typeof assistRef.current.hidePopup === 'function') {
assistRef.current.hidePopup();
}
} else if (args.command.label === 'Discard') {
if (assistRef.current && typeof assistRef.current.hidePopup === 'function') {
assistRef.current.hidePopup();
}
}
};
const handleToolbarItemClick = (args: ToolbarItemClickEventArgs): void => {
if (args.item && args.item.iconCss === 'e-icons e-inline-stop') {
setStopStreaming(true);
}
};
return (
<div>
<button
id="summarizeBtn"
className="e-btn e-primary"
style=
onClick={handleSummarizeBtnClick}
>
Content Summarize
</button>
<div
id="editableText"
contentEditable="true"
ref={editableRef}
>
<p>Inline AI Assist component provides intelligent text processing capabilities that enhance user productivity. It leverages advanced natural language processing to understand context and deliver precise suggestions. Users can seamlessly integrate AI-powered features into their applications.</p>
<p>With real-time response streaming and customizable prompts, developers can create interactive experiences. The component supports multiple response modes including inline editing and popup-based interactions.</p>
</div>
<InlineAIAssistComponent
id="defaultInlineAssist"
ref={assistRef}
relateTo="#summarizeBtn"
promptRequest={handlePromptRequest}
responseSettings=
inlineToolbarSettings=
popupWidth="500px"
/>
</div>
);
};
ReactDOM.render(<App />, document.getElementById('container'));#loader {
color: #008cff;
height: 40px;
left: 45%;
position: absolute;
top: 45%;
width: 30%;
}
#container {
margin: 20px auto;
}
#editableText {
width: 100%;
min-height: 120px;
max-height: 300px;
overflow-y: auto;
font-size: 16px;
padding: 12px;
border-radius: 4px;
border: 1px solid;
}