Practical LLM 3 - Extract Key Terms

In the third of Practical LLM series(See Part 1, Part 2), we will discuss how to build an app to extract key terms from an input text. This can be used to enhance user experience of an app by allowing user to enter a freeform text. The AI service will be used to extract inputs from the freemform text and the inputs can be fed to a database or service to present response to the user.

The architecture of this application is similar to our previous application to extract PDF content, consisting of a UI layer, Service Layer and AI Model. The UI Layer accepts a single user input, i.e. freeform text pharse and passes to the service. To demo the use case, we are also passing fields to be extracted, which in a real application can be configured.

Our front end code is quite simple. It simply allows uploading the file and POST it to the service. Relevant code is shown below:

    setIsLoading(true);
    setResult([]);
    try {
      const data = {extractMessage: {fields:fields, text:text}};
      const res = await fetch(serviceUrl, {
        method: 'POST',
        body: JSON.stringify(data)
      })
      // handle the error
      if (!res.ok) throw new Error(await res.text())
      setIsLoading(false);
      const resJ = await res.json();
      const msg2 = JSON.stringify(resJ).replaceAll('"','');
      const msg3 = msg2.trim();
      const msg4 = msg3.split('\\n');
      setResult(msg4);
    } catch (e: any) {
      // Handle errors here
      console.error(e)
      setIsLoading(false);
    }

JSON input payload to the service is given below:

{"extractMessage":
   {
    "fields":["company name","demographic","annual income","business type"],
   "text":"I want to learn about ABC inc. selling ice cream for school aged children and making 1M monthly income"
   }
}

Full source code for UI is available here. https://github.com/sekharkafle/extractui

Next lets discuss the backend. The backend code creates and sends the prompt to the AI model to extract key values from the input text:

const formatExtractMessage = (message) => {
  const txt = message.text;
    const fields = message.fields.join(', ');
    return `Human:You are an expert assistant with expertise in extracting key values. From the given text, extract ${fields}. If an extracted value is currency, return it as a whole number.\n<TEXT>${txt}</TEXT>\n\nAssistant:`;
};

export const handler = async (event)=> {
  dotenv.config();
  if (event) {
    const bedrock = new BedrockRuntimeClient({
      serviceId: 'bedrock',
      region: 'us-east-1',
    });
    let prompt = "";
    if(event.extractMessage){
      prompt = formatExtractMessage(event.extractMessage);
    const resText = await sendAIRequest(prompt);
    let aiRes = JSON.parse(resText)
    return aiRes.completion
  }
  return {
    statusCode: 400,
    body: `{'error':'Invalid input params.'}`,
  };
};

Here is the response from the service:

"Company name: ABC inc.\nDemographic: school aged children
\nAnnual income: 12000000\nBusiness type: selling ice cream"

In this application, the extracted response is in text format because we haven’t specified the response format in our prompt.

`Human:You are an expert assistant with expertise in extracting key values. 
From the given text, extract ${fields}. If an extracted value is currency,
 return it as a whole number.\n<TEXT>${txt}</TEXT>\n\nAssistant:

In a real world application, the extracted values may be used to query database or call a webservice to return more meaningful result to the user. So the response in json format would be more approipriate in that case. We can modify the prompt to respond data in json format.

Human:You are an expert assistant with expertise in extracting key values. 
From the given text, extract company name,demographic,annual income,business type.
 If an extracted value is currency, return it as a whole number.
  Response should be in JSON format with no other text in response.\n
  <TEXT>I want to learn about ABC inc. selling ice cream for school aged 
  children and making 1M monthly income</TEXT>\n\nAssistant:

Now the response looks more appropriate for consumption:

{
  "company_name": "ABC inc.",
  "demographic": "school aged children", 
  "annual_income": 12000000,
  "business_type": "selling ice cream"
}

Full source code for Lambda Service is available here. https://github.com/sekharkafle/extractlambda

With both back end and front end code ready to go, we are all set to test the web app:

Extract UI

Happy Extracting!!!