Bot Reports

The reports feature is for enterprise users to generate or schedule (generate periodically) specific reports from the available reports types and receive them over available delivery modes.

These reports can provide valuable information around user behavior, usage, engagement, product performance, etc. Users can have the desired information delivered to their email, SFTP path or S3 bucket.

Users can select the type of report from a list of pre-built reports. They can also select whether they want to generate a one-time report instantly or at regular intervals.

Configure

This section lists out all the reports that are currently active and generated periodically. The following details are available for the list of reports:

Clicking any report on this page lets you edit the configuration of previously created reports. This screen also has the "New Report" button which lets users create new reports.

Creating a report

Use the following procedure to create a bot report for Q&A, task and smart bots.

1. Navigate to the Reports tab from the bot dashboard. The Reports screen appears

2. Click New Report.

The Create a report subscription screen appears.

3. Specify this information to create and configure the report:

• Bot - The name of the bot associated with the report. You can view all the Q&A and Task bots created on the application.
• Report type - The pre-built report type the user wants to subscribe to. Selecting the Q&A bot in the Bot field displays three different types of reports:
  1. Usage and Behavior summary
  2. Message transcripts
  3. Curation issues
• Output format - The format in which the subscribed report is generated and sent to the configured delivery mode. Possible formats:
  1. csv
  2. xlsx

On-demand report

• Select the Generate now radio button
• Start date - Date indicating the start of the period user wants the data for.
• End date - Date indicating the end of the period user wants the data for.

📘

Note

The maximum duration for which an on-demand report can be generated is 30 days.

Scheduled report

• Select the Schedule report radio button
• Frequency - The often the report can be generated. Possible values:
  1. Daily
  2. Weekly
  3. Monthly
• Time - The time of day at which the subscriber receives the report at the set frequency.
• Time zone - The time zone for the configured time.

📘

Note

Time and time zone only dictate the delivery team for the reports. The data included in the report will be:

• From 00:00:00 to 23:59:59 for daily report.
• From 00:00:00 to 23:59:59 for last 7 days for a weekly report.
• From 00:00:00 to 23:59:59 for last 30 days for a monthly report.

• Start date - The date starting from which the subscriber starts receiving the report.

4. Select any of the following delivery modes and provide the required details.
   1. SFTP
   2. Email
   3. S3 bucket
5. Click Create to generate or subscribe to the report.

Updating an existing scheduled report

Once the report is created, the Enable report toggle gets added on the Edit a report subscription screen.

Enabling this toggle sends the report at the scheduled frequency and time and disabling this toggle stops you from receiving the report to the selected delivery mode.

📘

Note:

You can perform the following actions on the Edit a report subscription screen:

• Delete the report, using the Delete option
• Update the modified report details, using the Update option

History

The reports history screen lists out all the historical information of the reports dispatched till date. There is the option to download these historical reports if required by clicking the download icon under "Actions" column.

📘

Note

On-demand reports are populated in the history tab as soon as the users request for them but are available for download only once the generation is complete.

Report types and delivery modes

Users can choose from a list of pre-built reports based on the selected bot type. This section covers these report types, the sheets included in each report and the columns available in each sheet.

Q&A bot report types

There are three different report types available for Q&A bot in the bot builder application. As a bot developer, you can create different report types. These can be used to understand the bot usage summary, bot behavior, what users are asking and how bot is responding to the queries. You can also view the messages that ended-up as issues in curation.

Usage and behaviour summary

Displays the bot summary with the frequency at which articles and categories are invoked. You can view the summary, categories, and articles information in separate tab on the report.

Summary tab contains these details:

Bot name
The name of the bot
Total conversations
Total conversations /sessions handled by the bot
Conversations with at least one user messages
Conversations/sessions where users provided at least one input
Total human messages
The messages sent by end-users to the bot
Total bot responses
Total messages sent by the bot to end-users
Total partial matches
Cases where there was some ambiguity about user's message and bot responded with multiple intents as options
Conversations sent to agent
Total conversations handed over to a human agent
Total upvotes
Total bot responses that were upvoted by users
Total downvotes
Total bot responses that were downvoted by users

The categories tab displays these details
Category name
The name of the category as configured in the bot.
Conversations for the category
The number of conversations/sessions where an article belonging to this category was detected.
Total responses
The number of times an article belonging to this category was detected.
Total upvotes
The number of times a response from this category was upvoted.
Total downvotes
The number of times a response from this category was downvoted.

The Articles tab displays this information:
Article name
The name of the article (default variant) as configured in the bot
Article category
The category this intent belongs to
Conversations for the article
The number of conversations/sessions where this this article was detected
Total responses
The number of times where this article was detected
Total upvotes
The number of times the response for this article was upvoted
Total downvotes
The number of times a response for this article was downvoted

Message transcripts

Displays the conversation happened between the bot and consumer along with the similarity score. You can view these details in the report:
Timestamp
The timestamp for the message
Session ID
The unique identifier for the user's session
Consumer ID
The unique identifier for the end-user on Bot builder
Message type
The bot message or human message
Message text
The content of the message
Article
The identifier for the response sent back by the bot
Category
The intent detected by the bot for user's message
Top match score
The similarity score for the detected intent
Matched article 1
The intent detected by the selected NLU engine
Article 1 score
The score for the intent detected
Feedback
The user feedback if a message was upvoted or downvoted
Feedback comment
The comments left by users when downvoting a message

Curation issues

Displays the messages that ended up in curation as issues for various reasons . You can view these details in the report:

Timestamp
Timestamp for the message.
Session ID
Unique identifier for the user's session.
Consumer ID
Unique identifier for the end-user on Bot builder.
Human message
Content of the human message.
Bot message
Content of the message bot responded with.
Issue reason
The reason for this message ending up in curation.
Article
Identifier for the response sent back by the bot.
Category
Intent detected by the bot for user's message.
Top match score
Similarity score for the detected intent.
Matched article 1
Intent detected by the selected NLU engine.
Article 1 score
Score for the intent detected.

Task bot report types

There are three different report types available for Task bot in the bot builder application. As a bot developer, you can create different report types. These can be used to understand the bot usage summary, bot behavior, what users are asking and how bot is responding to the queries. You can also view the messages that ended-up as issues in curation.

Usage and behaviour summary

Displays the summary of conversations along with intents and template keys that are triggered. The
summary report displays these details:

Bot name
The name of the bot.
Total conversations
Total conversations/sessions handled by the bot.
Conversations with at least one user messages
Conversations/sessions where users provided at least one input.
Total human messages
Messages sent by end-users the bot.
Total bot responses
Total messages sent by the bot to end-users.
Total partial matches
Cases where there was some ambiguity about user's message and bot responded with multiple intents as options.
Conversations sent to agent
Total conversations handed over to a human agent.
Total upvotes
Total bot responses that were upvoted by users.
Total downvotes
Total bot responses that were downvoted by users.

You can also view the intent details in the Intents tab of the spreadsheet:

Intent name
The name of the intent as configured in the bot
Conversations for the intent
Number of conversations/sessions where this intent was invoked
Total invocations
Number of times this intent was invoked
Total completions
Number of times all the slots were collected and this intent was completed
Total upvotes
Total responses for that were upvoted for each intent
Total downvotes
Total responses for that were downvoted for each intent

The report also has high-level template details such as:

Template key name
Name of the template as configured in the bot
Template key intent
Intents where this template key is used
Conversations for the template key
Number of conversations/sessions where this template key was sent out as a response
Total responses
Number of times where this template key was sent out as a response
Total upvotes
Number of times the response for this template was upvoted
Total downvotes
Number of times a response for this template was downvoted

Messages transcripts

Displays the conversation of consumer with the bot along with the similarity scores. You can view
these details in the report:

Timestamp
Timestamp for the message
Session ID
Unique identifier for the user's session
Consumer ID
Unique identifier for the end-user on Bot builder
Message type
Bot message or human message
Message text
Content of the message
Template key
Identifier for the response sent back by the bot
Intent
Intent detected by the bot for user's message
Top match score
Similarity score for the detected intent
Matched intent 1
Intent detected by the selected NLU engine
Intent 1 score
Score for the intent detected
Feedback
User feedback if a message was upvoted or downvoted
Feedback comment
Comments left by users when downvoting a message.

Curation issues

Displays the messages that ended up in curation as issues for various reasons. You can view these
details in the report such as:

Timestamp
Timestamp for the message
Session ID
Unique identifier for the user's session
Consumer ID
Unique identifier for the end-user on Bot builder
Human message
Content of the human message
Bot message
Content of the message bot responded with
Issue reason
The reason for this message ending up in curation
Template key
Identifier for the response sent back by the bot
Intent
Intent detected by the bot for user's message
Top match score
Similarity score for the detected intent
Matched intent 1
Intent detected by the selected NLU engine
Intent 1 score
Score for the intent detected

Delivery modes

SFTP

Push reports to a secure location as scheduled
Enable this toggle to push the reports to the secure location at the scheduled time. You can only provide the following details by enabling this toggle.

IP Address
The IP address of the system.
Port
The port number to use for communication with the system.
Username
The username to access the reports.
Password
The password to access the reports.
Private key
The private key to access the files.
Upload path
The path where the files are routed in the system.

Email

Schedule emails for multiple recipients, separate with semi-colon(;)
Enable this toggle to add recipients.
Recipients
The email address of all recipients who must receive the reports at the specified time and frequency.

S3 bucket

Upload reports to a S3 bucket as per schedule
Enable this toggle to make the S3 fields available and route the reports to the configured S3 bucket.

AWS access key ID
The access key ID to access the AWS services and resources.
AWS secret access key
The secret access key to access the AWS services and resources.
Bucket name
The name of the bucket to which the report is routed.
Folder name
The name of the folder that is created in the S3 bucket.