Prototyping AI APIs using AWS

Prototyping AI APIs using AWS

Prototyping AI APIs using AWS

In the rapidly evolving world of Artificial Intelligence (AI), the ability to analyze and understand text data is becoming increasingly important. Amazon Web Services (AWS) offers a powerful service called Amazon Comprehend that allows developers to build Natural Language Processing (NLP) applications without the need for extensive machine learning expertise. In this blog post, we will explore the process of prototyping AI APIs using AWS Comprehend to extract valuable insights from text data.

Understanding AWS Comprehend

Amazon Comprehend is a fully managed NLP service that enables businesses to extract useful information and relationships from unstructured text data. It can identify entities, key phrases, sentiments, language, and more from a variety of text sources, including documents, social media, and customer feedback.

Getting Started: Setting Up AWS Comprehend:

AWS Account Setup: If you don't already have an AWS account, sign up and create one.
Access Amazon Comprehend: Navigate to the AWS Management Console and locate the Amazon Comprehend service.
Create a Custom IAM Role: Create an IAM role that grants necessary permissions to Amazon Comprehend.

Prototyping AI APIs using AWS Comprehend:

Text Sentiment Analysis:

Use the AWS SDK to interact with the Comprehend service programmatically.

  $ aws comprehend help
  NAME
         comprehend -

  DESCRIPTION
         Amazon Comprehend is an Amazon Web Services service for gaining insight
         into the content of documents. Use these actions to determine the  top-
         ics  contained in your documents, the topics they discuss, the predomi-
         nant sentiment expressed in them, the predominant  language  used,  and
         more.

  AVAILABLE COMMANDS
         o batch-detect-dominant-language

         o batch-detect-entities

         o batch-detect-key-phrases

         o batch-detect-sentiment

         o batch-detect-syntax

         o batch-detect-targeted-sentiment

         o classify-document

         o contains-pii-entities

         o create-dataset

         o create-document-classifier

         o create-endpoint

         o create-entity-recognizer

         o create-flywheel

         o delete-document-classifier

         o delete-endpoint

         o delete-entity-recognizer

         o delete-flywheel

         o delete-resource-policy

         o describe-dataset

Let's say we want to do the sentiment analysis of text - "I love Python Programming language"

  $ aws comprehend detect-sentiment --language-code "en" --text "I love Python programming language"
  {
      "Sentiment": "POSITIVE",
      "SentimentScore": {
          "Positive": 0.9986007809638977,
          "Negative": 9.051552478922531e-05,
          "Neutral": 0.0011894343188032508,
          "Mixed": 0.00011932779307244346
      }
  }

We can see the sentiment analysis is POSITIVE which is very true.

Now let's say we want to analyse a document or a data from website which is very simple. We will use lynx command to get the data from website into a variable.
```
  $ TEXT=`lynx -dump https://en.wikipedia.org/wiki/Python_\(programming_language\) | head -c 5000`
  $
```

Pass the $TEXT variable to a detect-sentiment command

  $ aws comprehend detect-sentiment --language-code "en" --text "$TEXT"
  {
      "Sentiment": "NEUTRAL",
      "SentimentScore": {
          "Positive": 0.00020325076184235513,
          "Negative": 0.0005437986110337079,
          "Neutral": 0.999243974685669,
          "Mixed": 9.053033863892779e-06
      }
  }

This is how Comprehend classifies text as positive, negative, neutral, or mixed sentiment.

Entity Recognition:

Detects named entities in input text when you use the pre-trained model. Detects custom entities if you have a custom entity recognition model.

When detecting named entities using the pre-trained model, use plain text as the input.

Let's say we want to analysis our website data and want to know about word count for each word.

  $ aws comprehend detect-entities --language-code "en" --text "$TEXT" --output text | cut -f 5 |sort|uniq -c |sort -nr -k 1
        4 Wikipedia
        2 Python
        2 Norsk
        2 English
        2 Bahasa
        2 한국어
        1 粵語
        1 日本語
        1 吴语
        1 中文
        1 සිංහල
        1 မြန်မာဘာသာ
        1 বাংলা
        1 অসমীয়া
        1 മലയാളം
        1 தமிழ்

Explore how Comprehend can enhance data extraction and organization.

Key Phrase Extraction:

Detects the key noun phrases found in the text.

  $ aws comprehend detect-key-phrases --language-code "en" --text "$TEXT" --output text
  KEYPHRASES      3       6       0.8861472606658936      #[1
  KEYPHRASES      7       29      0.7588788866996765      alternate [2]Wikipedia
  KEYPHRASES      35      57      0.8782252073287964      [3]Wikipedia Atom feed
  KEYPHRASES      62      69      0.7381764054298401      [4]Jump
  KEYPHRASES      73      80      0.5808557271957397      content
  KEYPHRASES      85      98      0.5795749425888062      [ ] Main menu
  KEYPHRASES      107     111     0.5609327554702759      menu
  KEYPHRASES      116     122     0.9981253743171692      BUTTON
  KEYPHRASES      141     147     0.9974844455718994      BUTTON
  KEYPHRASES      157     167     0.9399071335792542      Navigation
  KEYPHRASES      175     187     0.8160272836685181      [5]Main page
  KEYPHRASES      195     206     0.7837936878204346      [6]Contents
  KEYPHRASES      215     231     0.7777291536331177      7]Current events
  KEYPHRASES      239     256     0.858264684677124       [8]Random article
  KEYPHRASES      264     267     0.7898728847503662      [9]
  KEYPHRASES      273     282     0.8571096062660217      Wikipedia
  KEYPHRASES      290     294     0.7877490520477295      [10]
  KEYPHRASES      312     337     0.8975982069969177      [11]Donate

Use Case: Analyzing Customer Feedback

This kind of tool is very helpful, where a company can use AWS Comprehend to analyze customer feedback from various sources. They can improve customer satisfaction by doing sentiment analysis, entity recognition, and key phrase extraction.