Integrate LLM via Ollama With TypeScript Inline AI Assist control

2 Apr 202610 minutes to read

The Inline AI Assist control integrates with LLM via Ollama to enable advanced conversational AI features in your application. The control acts as a user interface where user prompts are sent to the selected LLM model via API calls, providing natural language understanding and context-aware responses.

Prerequisites

Before starting, ensure you have the following:

Node.js: Version 16 or higher with npm.
Ollama installed to run and manage LLM models locally.
Syncfusion Inline AI Assist: Package @syncfusion/ej2-interactive-chat installed.
Marked Library: For parsing Markdown responses (npm install marked --save).

Set Up the Environment

Follow the Getting Started guide to configure and render the Inline AI Assist control in your application.

Install Dependency

To install the marked library, run npm install marked --save in your project directory to add it as a dependency in your package.json file.

Configuring Ollama

Install Ollama for your operating system:

Windows
macOS
main.ts
1. Visit [Windows](https://ollama.com/download)
2. Click `Download for Windows` to get the `.exe installer`. 
3. Run `OllamaSetup.exe` and follow the wizard to install.
1. Visit [macOS](https://ollama.com/download/mac)
2. Click `Download for macOS` to get `.dmg file`
3. Install it by following the wizard.
1. Visit [Linux](https://ollama.com/download/linux)
2. Run the below command to install Ollama in your system 

```bash
          
curl -fsSL https://ollama.ai/install.sh | sh

```

Download and run an Ollama model

Download and run a model using the following command. Replace deepseek-r1 with your preferred model (e.g., llama3, phi4). See the Ollama model library for available models.

ollama run deepseek-r1

After the model download completes, start the Ollama server to make the model accessible:

ollama serve

Configure Inline AI Assist with Ollama

Modify the index.js file to integrate the Ollama model with the Inline AI Assist control .

index.ts
index.html
import { InlineAIAssist, InlinePromptRequestEventArgs } from "@syncfusion/ej2-interactive-chat";
import marked from 'marked';

let stopStreaming: boolean = false;

// Initialize Inline AI Assist
const inlineAIAssist = new InlineAIAssist({
  promptRequest:onPromptrequest,
  inlineToolbarSettings: {
    itemClick: (args: any) => {
      if (args.item.iconCss === 'e-icons e-inline-stop') {
          handleStopResponse();
      }
    }
  },
  relateTo: '#summarizeBtn',
  responseSettings: {
    itemSelect: (args: any): void => {
      if (args.command.label === 'Accept') {
          const editable = document.getElementById('editableText') as HTMLElement | null;
          if (editable) {
              editable.innerHTML = '<p>' + inlineAIAssist.prompts[inlineAIAssist.prompts.length - 1 ].response + '</p>';
          }
          inlineAIAssist.hidePopup();
      } else if (args.command.label === 'Discard') {
          inlineAIAssist.hidePopup();
      }
    }
  }
});

//  Handle user prompt: call local LLM via Ollama
async function onPromptrequest(args: InlinePromptRequestEventArgs) {
  let lastResponse = "";
  const defaultResponse = "⚠️ Something went wrong while connecting to the AI service. Please check your Ollama application running background.";
  try {
      // Send request to Ollama API
      const response = await fetch('http://localhost:11434/api/generate', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
          model: 'deepseek-r1',
          prompt: `### Instruction:\nRespond in up to 5 lines.\n\n### Input:\n${args.prompt}`,
          stream: false,
      }),
      });
      const reply = await response.json();
      const responseUpdateRate = 10;

      // Stream AI response in chunks
      async function streamResponse(response:string) {
          let i = 0;
          const responseLength = response.length;
          while (i < responseLength && !stopStreaming) {
          lastResponse += response[i];
          i++;
          if (i % responseUpdateRate === 0 || i === responseLength) {
              const htmlResponse =marked.parse(lastResponse);
              inlineAIAssist.addResponse(htmlResponse, i === responseLength);
          }
          await new Promise(resolve => setTimeout(resolve, 15)); // Delay before the next chunk
      }
 }
  if(response) {
  stopStreaming = false;
  streamResponse(reply.response);
  }

  } catch (error) {
      inlineAIAssist.addResponse(defaultResponse, true);
  }
}

function handleStopResponse(): void {
  stopStreaming = true;
}

// Append the Inline AI Assist to the container
inlineAIAssist.appendTo('#defaultInlineAssist');
const summarizeBtn: HTMLElement = document.querySelector('#summarizeBtn') as HTMLElement;
if (summarizeBtn) {
    summarizeBtn.addEventListener('click', () => {
        inlineAIAssist.showPopup();
    });
}
<!DOCTYPE html>
<html lang="en">

<head>
    <title>EJ2 Inline AI Assist</title>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta name="description" content="TypeScript Inline AI Assist Control" />
    <meta name="author" content="Syncfusion" />
    <link href="index.css" rel="stylesheet" />
    <link href="https://cdn.syncfusion.com/ej2/33.2.3/ej2-base/styles/tailwind3.css" rel="stylesheet" />
    <link href="https://cdn.syncfusion.com/ej2/33.2.3/ej2-interactive-chat/styles/tailwind3.css" rel="stylesheet" />
    <link href="https://cdn.syncfusion.com/ej2/33.2.3/ej2-inputs/styles/tailwind3.css" rel="stylesheet" />
    <link href="https://cdn.syncfusion.com/ej2/33.2.3/ej2-buttons/styles/tailwind3.css" rel="stylesheet" />
    <link href="https://cdn.syncfusion.com/ej2/33.2.3/ej2-navigations/styles/tailwind3.css" rel="stylesheet" />
    <link href="https://cdn.syncfusion.com/ej2/33.2.3/ej2-notifications/styles/tailwind3.css" rel="stylesheet" />
    <script src="https://cdnjs.cloudflare.com/ajax/libs/systemjs/0.19.38/system.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
    <script src="systemjs.config.js"></script>
    <style>
        #editableText {
            width: 100%;
            min-height: 120px;
            max-height: 300px;
            overflow-y: auto;
            font-size: 16px;
            padding: 12px;
            border-radius: 4px;
            border: 1px solid;
        }
    </style>
</head>
<body>
    <div id='loader'>Loading....</div>
    <div id='container' style="height: 350px; width: 650px;">
        <button id="summarizeBtn" class="e-btn e-primary" style="margin-bottom: 10px;">Content Summarize</button>
        <div id="editableText" contenteditable="true">
            <p>Inline AI Assist component provides intelligent text processing capabilities that enhance user productivity. It leverages advanced natural language processing to understand context and deliver precise suggestions. Users can seamlessly integrate AI-powered features into their applications.</p>
            <p>With real-time response streaming and customizable prompts, developers can create interactive experiences. The component supports multiple response modes including inline editing and popup-based interactions.</p>
        </div>
        <div id="defaultInlineAssist"></div>
    </div>
</body>
</html>

Search docs

Ask Syncfusion AI Assistant