Objective

In this project, we would be combining multiple services and open-source tools to make a Chatbot that recommends songs based on the tone of the conversation which the user is having with the chatbot.

Project Context

The purpose of chat bots is to support and scale business teams in their relations with customers. It could live in any major chat applications like Facebook Messenger, Slack, Telegram, Text Messages, etc. Chatbot applications streamline interactions between people and services, enhancing customer experience. At the same time, they offer companies new opportunities to improve the customers engagement process and operational efficiency by reducing the typical cost of customer service. This project is focussed on building a custom chatbot that will be your fundamental step of the learning curve of building your own professional chatbots.


But you must be tired of the weird chat bots out there in the world which are made for mainly business purposes? In this project, we would be building an extensive Chatbot service, to which you can talk to. And talking to a chatbot wouldn't be business-driven. It would just be casual conversations. Further, on top of it, the chatbot would also be recommending songs to the user based on the tone of the user. This song recommendation feature employs the use of Last.fm API, very much similar to the popular Spotify API. Also for tone/emotion analysis of the conversation we will be using the IBM Tone Analyzer API.


Collaborating with these types of APIs is very much critical as in today's world the popular chatbots do much more than simply having a data-driven conversation; to supplement additional user-oriented features. Also the reason to choose python to build the chatbot is because python boasts a wide array of open-source libraries for chatbots, including scikit-learn and TensorFlow. It is great for small data sets and more simple analyses; also Python's libraries are much more practical.

Product Architecture

image-20210110154629563

High-Level Approach

  • User starts the conversation
  • Emotional Analysis of the conversation is done using the IBM Emotional API
  • Get the reply to the conversation from the Cakechat Chatbot
  • Based on the Emotion which the app perceives, top songs are retrieved using Last.fm songs API
  • If a user listens to a particular song for sometime, a similar song would be recommended to the user using Last.fm API.

Primary goals

  • Setting up an open-source project locally and handling the errors being faced
  • Using multiple services to build up a new service over them.
  • Having a real-world chatbot, to which you can literally chat like you chatting to a real person and enjoying the music recommended by the system.

Objective

In this project, we would be combining multiple services and open-source tools to make a Chatbot that recommends songs based on the tone of the conversation which the user is having with the chatbot.

Project Context

The purpose of chat bots is to support and scale business teams in their relations with customers. It could live in any major chat applications like Facebook Messenger, Slack, Telegram, Text Messages, etc. Chatbot applications streamline interactions between people and services, enhancing customer experience. At the same time, they offer companies new opportunities to improve the customers engagement process and operational efficiency by reducing the typical cost of customer service. This project is focussed on building a custom chatbot that will be your fundamental step of the learning curve of building your own professional chatbots.


But you must be tired of the weird chat bots out there in the world which are made for mainly business purposes? In this project, we would be building an extensive Chatbot service, to which you can talk to. And talking to a chatbot wouldn't be business-driven. It would just be casual conversations. Further, on top of it, the chatbot would also be recommending songs to the user based on the tone of the user. This song recommendation feature employs the use of Last.fm API, very much similar to the popular Spotify API. Also for tone/emotion analysis of the conversation we will be using the IBM Tone Analyzer API.


Collaborating with these types of APIs is very much critical as in today's world the popular chatbots do much more than simply having a data-driven conversation; to supplement additional user-oriented features. Also the reason to choose python to build the chatbot is because python boasts a wide array of open-source libraries for chatbots, including scikit-learn and TensorFlow. It is great for small data sets and more simple analyses; also Python's libraries are much more practical.

Product Architecture

image-20210110154629563

High-Level Approach

  • User starts the conversation
  • Emotional Analysis of the conversation is done using the IBM Emotional API
  • Get the reply to the conversation from the Cakechat Chatbot
  • Based on the Emotion which the app perceives, top songs are retrieved using Last.fm songs API
  • If a user listens to a particular song for sometime, a similar song would be recommended to the user using Last.fm API.

Primary goals

  • Setting up an open-source project locally and handling the errors being faced
  • Using multiple services to build up a new service over them.
  • Having a real-world chatbot, to which you can literally chat like you chatting to a real person and enjoying the music recommended by the system.

Setting up the CakeChat Chatbot Server locally

Since Chatbot is the main part of the project, we would be setting up the Chatbot first. This part alone would give you a strong understanding of the project.


In this milestone, you need to install Cakechat Chatbot from this GitHub Repository.

Requirements

  • Create a virtual environment of python-3.
  • Fork and Clone the GitHub repository onto your local machine.
  • Install the dependencies as specified here.
  • Load the pre-trained model by following the steps specified here.
  • Run and test the Cakechat Server as specified here

Tip

  • If you see the following error :-

    ModuleNotFoundError: No module named 'boto3'
    

    Install the python dependency boto3 by using the following command:-

    pip install boto3 
    
  • If you see the following error :-

    AttributeError: 'str' object has no attribute 'decode'
    

    have a look here.

Expected Outcome

  • By the end, you will be having a running Cakechat Chatbot Server on your system and you would be able to get a response based on a conversation like shown below:-
python tools/test_api.py -f 127.0.0.1 -p 8080 \
>     -c "Hi, Eddie, what's up?" \
>     -c "Not much, what about you?" \
>     -c "Fine, thanks. Are you going to the movies tomorrow?"
Using TensorFlow backend.
{'response': "Yes. I was going to go but didn't know you're coming!"}
  • Further, on testing it using Postman, the output looks like this:-

image-20210103025904180

Setting Up the IBM Tone Analyzer API

In this milestone, we would be setting up the IBM Tone Analyzer API so that we can analyze the tone of conversation (emotions). We are using an API here as we don't have that enough data, computational power and time to create our own model API. This milestone will make you realise why to prefer using open-source APIs rather than creating your own models each time.

Requirements

  • Check this website to have an idea of what IBM Tone Analyzer API would be doing.
  • Create an account on IBM Cloud (It's free)
  • Enable the Tone Analyzer Service for your account from here.
  • Try running the Python code for analyzing the conversation from here and don't forget to replace {apikey} and {url} with the apikey and url you received by enabling Tone Analyser Service for your account.

Tip

  • If you see any error like

    jwt.exceptions.DecodeError: It is required that you pass in a value for the "algorithms" argument when calling decode().
    

    Then have a look here and install the following dependency

    pip install PyJWT==1.7.1
    

Expected Outcome

  • After running the code, tone Analyser Service, you would have the analysis of the text on your system which looks similar to this. Here as the top is the tone of the entire app and below is the tone of every sentence.
  {
    "document_tone": {
      "tones": [
        {
          "score": 0.6165,
          "tone_id": "sadness",
          "tone_name": "Sadness"
        },
        {
          "score": 0.829888,
          "tone_id": "analytical",
          "tone_name": "Analytical"
        }
      ]
    },
    "sentences_tone": [
      {
        "sentence_id": 0,
        "text": "Team, I know that times are tough!",
        "tones": [
          {
            "score": 0.801827,
            "tone_id": "analytical",
            "tone_name": "Analytical"
          }
        ]
      },
      {
        "sentence_id": 1,
        "text": "Product sales have been disappointing for the past three quarters.",
        "tones": [
          {
            "score": 0.771241,
            "tone_id": "sadness",
            "tone_name": "Sadness"
          },
          {
            "score": 0.687768,
            "tone_id": "analytical",
            "tone_name": "Analytical"
          }
        ]
      },
      {
        "sentence_id": 2,
        "text": "We have a competitive product, but we need to do a better job of selling it!",
        "tones": [
          {
            "score": 0.506763,
            "tone_id": "analytical",
            "tone_name": "Analytical"
          }
        ]
      }
    ]
  }

So, now for any given phrase or sentence, you can analyze the tone of that. We would be using this code ahead to analyze the tone of the conversation.

Setting up Last.fm Songs API and testing it

In this milestone, we would be setting up the Last.fm Songs API so that we can recommend some songs to the user based on the tone/emotion of the user. We are using an API here as we don't have enough data, computational power and time to scrape the web for songs based on the specific information of tones we have extracted previously.

Requirements

  • Create API account on Last.fm from here and get the API_KEY
  • Test the following API's features with the help of your API_KEY
    • Getting top 5 songs of a particular tag
    • Getting similar songs related to a song

Note

Work with JSON API's only and not XML API

Expected Outcome

  • By the end of this milestone, you would have seen the response of both the above-specified API's

    • Getting top 5 songs of a particular tag

    image-20210110193356469

    • Get similar songs related to a given song id

    image-20210110205710676

Tip

Try using mbid instead of just using track name and artist for finding songs similar to a given song.

Understanding the overall process involved in the chatbot's working

Once we are done with this milestone, we would be having a very good understanding of the complete project architecture. Further, after this, you can take this project in a new direction after this milestone. Completing this milestone is critical as there are a lot of components that need to be integrated together to complete the chatbot; having a clear sense of product architecture is needed here.

  • Have a look at the High-Level approach which we briefed earlier

    High-Level Approach
      
    - User starts the conversation
    - Emotional Analysis of the conversation is done using the IBM Emotional API  
    - Get the reply to the conversation from the Cakechat Chatbot 
    - Based on the Emotion which we got, top songs are retrieved using Last.fm songs API
    - If a user listens to a particular song ... a similar song to that song would be recommended to the user using Last.fm API. 
    

We would be implementing this approach manually to get started right away by buidling the chatbot in the next milestone.

Requirements

  • Let's start the conversation with a good mood tone:
  Hey there!! What's up? How's the day?
  • Now let's send this text for Emotional Analysis. And we get the following response:
  {
    "document_tone": {
      "tones": [
        {
          "score": 0.66525,
          "tone_id": "joy",
          "tone_name": "Joy"
        }
      ]
    },
    "sentences_tone": [
      {
        "sentence_id": 0,
        "text": "Hey there!! What's up?",
        "tones": []
      },
      {
        "sentence_id": 1,
        "text": "How's the day?",
        "tones": []
      }
    ]
  }
  • Now send the request to the chatbot server with the emotion of chatbot using the following mapping:
  {
      "anger": "anger",
      "fear":"sadness",
      "joy":"joy",
      "sadness":"sadness",
      "analytical":"neutral",
      "confident":"neutral",
      "tentative":"neutral"
  }

Why this mapping? Because the Cakechat chatbot supports conversation in limited emotions only.

Just wondering... how can you have an analytical conversation? ... but still, you can have an analytical tone ... Therefore we are mapping Analytical tone to Neutral conversation. 

So, using the above mapping we would be talking to the server.

  • Now, send the request to the cakechat server with the conversation and the emotion of the chat.

    Request:

    {
        "context": [
            "Hey there!! What's up? How's the day?"
        ],
        "emotion": "joy"
    } 
    

    Response:

    {
        "response": "It's been good. How about you?"
    }
    

    As you can see we got the response. So, you can continue further with the conversation appended in the context variable.

  • Now, it's time to get a song recommendation. Have a look and analyze the number of songs of a particular tag using last.fm tag.gettoptracks API.

    {
        "tracks": {
            "track": [
                {...},
                {...}
            ],
            "@attr": {
                "tag": "analytical",
                "page": "1",
                "perPage": "5",
                "totalPages": "1",
                "total": "2"
            }
        }
    }
    

    You would have seen here also that we were only able to find 2 songs corresponding to the analytical tag. So, instead of using the tags returned by the Emotional API, we can map them using the mapping defined earlier above.

  • Now, once a user would click on a particular song, we also need to recommend similar songs to that particular song. This can also be done using the last.fm's track.getsimilar API using the track and the artist of the song.

Expected Outcome

By now, you have a complete system and a good understanding of the project's flow, which is satisfied by the above requirements.

Putting everything together and build the chatbot

Once you have understood the complete flow, you can do whatever you want now. You can go forward with building a command-line interface that collects all these things and does them. Or you can build a separate Flask/Django server that can do the above steps as a part of an API or a Web Page.


As a part of this milestone, we would be continuing with building a Django/Flask server.

Requirements

  • Building the following API's

    • /tone
      • input:
        • the context required for the chatbot
      • output: the tone-analysis of the conversation
    • /response
      • input:
        • the context required for the chatbot
        • the tone from /tone
      • output: the chatbot response
    • /songs
      • input:
        • the tag for getting songs from last.fm API
      • output:
        • list of songs
    • /simmilarsongs
      • input:
        • the song name
        • the artist name
      • output:
        • list of songs
  • We can make a template inside your Django/flask app which has a page similar to this:

image-20210111022554235

Here, on the left, I have the songs recommended to the user, and on the right-hand side are the songs which were recommended based on one of the songs which I clicked on.

Further, the emotion of the chat is shown at the top

Tip

Context of the Cakechat Chatbot is: at max last 3 conversations.

Note

You can do this milestone in your own style.

Bring it On!

Further, you can learn about docker and build a docker-compose file that provides all the services in one place.

Expected Outcome

By the end of this milestone, you would have a working chatbot system with a good UI that recommends songs to the user along with chatting.