Prototyping AI APIs using AWS

In the rapidly evolving world of Artificial Intelligence (AI), the ability to analyze and understand text data is becoming increasingly important. Amazon Web Services (AWS) offers a powerful service called Amazon Comprehend that allows developers to build Natural Language Processing (NLP) applications without the need for extensive machine learning expertise. In this blog post, we will explore the process of prototyping AI APIs using AWS Comprehend to extract valuable insights from text data.

Understanding AWS Comprehend

Amazon Comprehend is a fully managed NLP service that enables businesses to extract useful information and relationships from unstructured text data. It can identify entities, key phrases, sentiments, language, and more from a variety of text sources, including documents, social media, and customer feedback.

Getting Started: Setting Up AWS Comprehend:

1. AWS Account Setup: If you don't already have an AWS account, sign up and create one.

2. Access Amazon Comprehend: Navigate to the AWS Management Console and locate the Amazon Comprehend service.

3. Create a Custom IAM Role: Create an IAM role that grants necessary permissions to Amazon Comprehend.

Prototyping AI APIs using AWS Comprehend:

1. Text Sentiment Analysis:

   • Use the AWS SDK to interact with the Comprehend service programmatically.

   •   $ aws comprehend help
       NAME
              comprehend -
     
       DESCRIPTION
              Amazon Comprehend is an Amazon Web Services service for gaining insight
              into the content of documents. Use these actions to determine the  top-
              ics  contained in your documents, the topics they discuss, the predomi-
              nant sentiment expressed in them, the predominant  language  used,  and
              more.
     
       AVAILABLE COMMANDS
              o batch-detect-dominant-language
     
              o batch-detect-entities
     
              o batch-detect-key-phrases
     
              o batch-detect-sentiment
     
              o batch-detect-syntax
     
              o batch-detect-targeted-sentiment
     
              o classify-document
     
              o contains-pii-entities
     
              o create-dataset
     
              o create-document-classifier
     
              o create-endpoint
     
              o create-entity-recognizer
     
              o create-flywheel
     
              o delete-document-classifier
     
              o delete-endpoint
     
              o delete-entity-recognizer
     
              o delete-flywheel
     
              o delete-resource-policy
     
              o describe-dataset
     

   • Let's say we want to do the sentiment analysis of text - "I love Python Programming language"

       $ aws comprehend detect-sentiment --language-code "en" --text "I love Python programming language"
       {
           "Sentiment": "POSITIVE",
           "SentimentScore": {
               "Positive": 0.9986007809638977,
               "Negative": 9.051552478922531e-05,
               "Neutral": 0.0011894343188032508,
               "Mixed": 0.00011932779307244346
           }
       }
     

     We can see the sentiment analysis is POSITIVE which is very true.

   • Now let's say we want to analyse a document or a data from website which is very simple. We will use lynx command to get the data from website into a variable.

       $ TEXT=`lynx -dump https://en.wikipedia.org/wiki/Python_\(programming_language\) | head -c 5000`
       $
     

   • Pass the $TEXT variable to a detect-sentiment command

       $ aws comprehend detect-sentiment --language-code "en" --text "$TEXT"
       {
           "Sentiment": "NEUTRAL",
           "SentimentScore": {
               "Positive": 0.00020325076184235513,
               "Negative": 0.0005437986110337079,
               "Neutral": 0.999243974685669,
               "Mixed": 9.053033863892779e-06
           }
       }
     

   • This is how Comprehend classifies text as positive, negative, neutral, or mixed sentiment.

1. Entity Recognition:

   • Detects named entities in input text when you use the pre-trained model. Detects custom entities if you have a custom entity recognition model.

     When detecting named entities using the pre-trained model, use plain text as the input.

   • Let's say we want to analysis our website data and want to know about word count for each word.

       $ aws comprehend detect-entities --language-code "en" --text "$TEXT" --output text | cut -f 5 |sort|uniq -c |sort -nr -k 1
             4 Wikipedia
             2 Python
             2 Norsk
             2 English
             2 Bahasa
             2 한국어
             1 粵語
             1 日本語
             1 吴语
             1 中文
             1 සිංහල
             1 မြန်မာဘာသာ
             1 বাংলা
             1 অসমীয়া
             1 മലയാളം
             1 தமிழ்
     

   • Explore how Comprehend can enhance data extraction and organization.

2. Key Phrase Extraction:

   • Detects the key noun phrases found in the text.

       $ aws comprehend detect-key-phrases --language-code "en" --text "$TEXT" --output text
       KEYPHRASES      3       6       0.8861472606658936      #[1
       KEYPHRASES      7       29      0.7588788866996765      alternate [2]Wikipedia
       KEYPHRASES      35      57      0.8782252073287964      [3]Wikipedia Atom feed
       KEYPHRASES      62      69      0.7381764054298401      [4]Jump
       KEYPHRASES      73      80      0.5808557271957397      content
       KEYPHRASES      85      98      0.5795749425888062      [ ] Main menu
       KEYPHRASES      107     111     0.5609327554702759      menu
       KEYPHRASES      116     122     0.9981253743171692      BUTTON
       KEYPHRASES      141     147     0.9974844455718994      BUTTON
       KEYPHRASES      157     167     0.9399071335792542      Navigation
       KEYPHRASES      175     187     0.8160272836685181      [5]Main page
       KEYPHRASES      195     206     0.7837936878204346      [6]Contents
       KEYPHRASES      215     231     0.7777291536331177      7]Current events
       KEYPHRASES      239     256     0.858264684677124       [8]Random article
       KEYPHRASES      264     267     0.7898728847503662      [9]
       KEYPHRASES      273     282     0.8571096062660217      Wikipedia
       KEYPHRASES      290     294     0.7877490520477295      [10]
       KEYPHRASES      312     337     0.8975982069969177      [11]Donate
     

Use Case: Analyzing Customer Feedback

This kind of tool is very helpful, where a company can use AWS Comprehend to analyze customer feedback from various sources. They can improve customer satisfaction by doing sentiment analysis, entity recognition, and key phrase extraction.