Knowledge connectors BETA

Knowledge connectors complement defined intents by parsing documents (FAQs or articles) to find questions and responses. Questions are added as training phrases and answers are added as responses. To configure them, you define one or more knowledge bases, which are collections of documents.

You can enable knowledge bases for your agent, so all user requests may find automated responses using your knowledge bases. Alternatively, you can specify one or more knowledge bases in your individual detectIntent requests.

It's common for an agent using knowledge connectors to also use defined intents, as knowledge connectors offer less response precision and control than intents. When using both intents and knowledge connectors, you should define your intents to handle complex user requests that require special handling and precision, and let knowledge connectors handle simple requests with responses automatically extracted from your documents. When you identify content in FAQs that users may want to know more about, you can convert questions into explicit intents, which gives you full control.

Things to consider:

  • This is a beta feature and is only available for agents using V2 of the API with beta features enabled. Enable beta features in the General settings tab.
  • Knowledge connectors are currently only available for English and related locale languages.

Create a knowledge base

A knowledge base contains information about the documents it's supplied. Dialogflow uses this information when looking for responses to user requests. To create a knowledge base, follow these steps:

Dialogflow UI

  1. Click on Knowledge in the left menu.

  1. Click CREATE KNOWLEDGE BASE.
  2. Enter a name for the knowledge base.
  3. Click SAVE.

cURL

When using this cURL command to create a knowledge base, replace the following information:

  • my-knowledge-base-display-name with a name of your choosing.
  • my-gcp-project with your GCP project ID.
curl \
-X POST \
-H "authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
--data "{
  'displayName': 'my-knowledge-base-display-name'
}" \
"https://dialogflow.googleapis.com/v2beta1/projects/my-gcp-project/knowledgeBases"

The response should look something like this:

{
  "name": "prejects/my-gcp-project/knowledgeBases/NDA4MTM4NzE2MjMwNDUxMjAwMA",
  "diaplayName": "my-knowledge-base-display-name"
}

The value for name is the knowledge base ID that you use for subsequent requests.

For more information on managing knowledge bases, see KnowledgeBases.

Add a document to the knowledge base

Knowledge bases can take FAQs or articles and parse the information to generate the content for your knowledge base.

Dialogflow UI

  1. Click on Knowledge in the left menu.
  2. Click on Create the first one or New Document.
  3. Enter a name for the document.
  4. Select text/html from the Mime Type list.
  5. Select FAQ from the Knowledge Type list.
  6. Select URL for the Data Source.
  7. Enter a FAQ or article URL for your knowledge base in the URL text field.
  8. Click CREATE.

cURL

When using this cURL command to add a document, replace the following information:

  • my-document-display-name with a name of your choosing.
  • my-gcp-project with your GCP project ID.
  • my-knowledge-base-id with your knowledge base ID.
curl \
-X POST \
-H "authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
--data "{
  'displayName': 'my-document-display-name',
  'mimeType': 'text/html',
  'knowledgeTypes': 'FAQ',
  'contentUri': 'https://dialogflow.com/v2-faq'
}" \
"https://dialogflow.googleapis.com/v2beta1/projects/my-gcp-project/knowledgeBases/my-knowledge-base-id/documents"

The response should look something like this:

{
"name": "projects/my-gcp-project/operations/ks-add_document-MzA5NTY2MTc5Mzg2Mzc5NDY4OA"
}

The value for name is the operation ID that you use for subsequent requests.

Creating a document may take some time to complete, depending on the size of the document. You can poll the status of the operation with a cURL command. When using this command, replace the following information:

  • my-gcp-project with your GCP project ID.
  • my-operation-id with the operation ID provided in the previous output.
curl \
-X GET \
-H "authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
"https://dialogflow.googleapis.com/v2beta1/projects/my-gcp-project/operations/my-operation-id"

Once the operation is complete, the response will have a true value for the donefield:

{
"name": "projects/my-gcp-project/operations/ks-add_document-MzA5NTY2MTc5Mzg2Mzc5NDY4OA",
"metadata": {
  "@type": "type.googleapis.com/google.cloud.dialogflow.v2beta1.KnowledgeOperationMetadata",
  "state": "DONE"
},
"done": true,
...
}

For more information on managing documents, see Documents.

See the Supported content section below for more details on options.

Knowledge connector settings

Click Knowledge in the left menu for knowledge base settings.

You can enable one or more knowledge bases for your agent. All user requests use enabled knowledge bases; however, a specific knowledge base can be included in a detectIntent request to provide a response from that set of answers.

To enable or disable knowledge bases:

  1. Check the corresponding box for a listed knowledge base.
  2. Click ENABLE or DISABLE to enable or disable the selected knowledge base.

ADJUST KNOWLEDGE RESULTS PREFERENCE is a value that changes the likelihood of a user's request being matched to either an intent or a knowledge base.

Rich responses and integrations

Knowledge connectors can be used with rich responses and integrations. To add knowledge connectors to responses, follow these steps:

  1. When editing an intent, scroll down to the Responses section and add a response.
  2. Use $Knowledge.Question[1] and $Knowledge.Answer[1] where you want the first question and answer to appear.

  3. Click SAVE.

When defining a response, consider the following points:

  • If the number of defined responses is greater than the number N of knowledge connector response matches, only N responses will be returned.
  • Given that the accuracy could be lower than explicitly matching defined intents, we recommend returning three or more responses to users when possible.

detectIntent with knowledge base

When performing a detectIntent request, you can specify one or more knowledge bases for a possible response. The example below specifies one knowledge base and uses the knowledge base that was set up in the previous steps.

When using this cURL command to perform detectIntent, replace the following information:

  • my-gcp-project with your GCP project ID.
  • my-knowledge-base-id with your knowledge base ID.
curl \
-X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
--data "{
 'queryInput': {
   'text': {
     'text': 'Which Dialogflow Edition is right for me?',
     'languageCode': 'en-US'
   }
 },
 'queryParams': {
   'knowledgeBaseNames': ['projects/my-gcp-project/knowledgeBases/my-knowledge-base-id']
 }
}" \
"https://dialogflow.googleapis.com/v2beta1/projects/my-gcp-project/agent/sessions/123456789:detectIntent"

For more information on request options, see detectIntent.

The response should look something like this:

{
  ...
  "queryResult": {
    "queryText": "Which Dialogflow Edition is right for me?",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "SWe anticipate that Dialogflow Standard Edition will remain the preferred option for the majority of use cases. Dialogflow Enterprise Edition is ideal for businesses that need Google Cloud Platform’s enterprise-grade reliability guarantees, compliance and customer support. For more information about different editions, see Dialogflow Editions.",
    "intent": {
      "name": "projects/my-gcp-project/agent/intents/487c7242-a769-408a-a339-47b95e10dac4",
      "displayName": "Knowledge.KnowledgeBase.MzkzNTAyMDE3NDQxNDk3MDg4MA"
    },
    "intentDetectionConfidence": 1,
    "languageCode": "en",
    "knowledgeAnswers": {
      "answers": [
        {
          "source": "projects/my-gcp-project/knowledgeBases/MTA5NzQ3MDkzNDE5NDg0Nzc0NDA/documents/NTc0MTI0NTA5OTk2NzI1MDQzMg",
          "faqQuestion": "Which Dialogflow Edition is right for me?",
          "answer": "We anticipate that Dialogflow Standard Edition will remain the preferred option for the majority of use cases. Dialogflow Enterprise Edition is ideal for businesses that need Google Cloud Platform’s enterprise-grade reliability guarantees, compliance and customer support. For more information about different editions, see Dialogflow Editions.",
          "matchConfidenceLevel": "HIGH",
          "matchConfidence": 1
        }
      ]
    }
  }
}

detectIntent responses

The response for a detectIntent request is a DetectIntentResponse message. Multiple factors affect how response fields are populated.

If a defined intent and a knowledge base are both potential matches, the match confidence of each is used to determine which is matched. The selected match is populated in the DetectIntentResponse.queryResult field, and other potential matches are populated in the DetectIntentResponse.alternativeQueryResults field.

If a knowledge base provides a potential match:

  • QueryResult.knowledgeAnswers is populated with a list of potential knowledge answers ordered by decreasing match confidence.
  • If rich responses have been defined for the knowledge base, QueryResult.fulfillmentMessages is populated with rich response messages.

When performing a detectIntent request, it is possible for the knowledge query to fail. When this happens, the agent selects defined intents, so the overall detectIntent request will not fail. You can find knowledge query error information in the DetectIntentResponse.alternativeQueryResults[i].diagnosticInfo field.

Supported content

The following table shows the supported MIME types by Knowledge Type and Source:

Knowledge Type / Source Uploaded file File from Cloud Storage File from public URL
FAQ text/csv, text/html text/csv, text/html text/html
Knowledge base article text/plain, text/html text/plain, text/html, application/pdf N/A

Known issues and limitations with adding document content

FAQ:

  • CSV must have questions in the first column and answers in the second with no header.
  • CSV should be used whenever possible, as they are parsed more accurately.
  • HTML content from a private Cloud Storage file is not supported.
  • Public HTML content with a single question/answer pair is not supported.

Knowledge base article:

  • Knowledge base article is currently experimental. It is based on similar technologies that have been tried and tested at Google in products like Search and Assistant.
  • Content with dense text works best. Avoid content with many single sentence paragraphs.
  • Tables and lists are not supported.

General:

  • Dialogflow removes HTML tags from content when creating responses.
  • Google Assistant responses have a 640 character limit per chat bubble. Long answers will be truncated when integrating with Google Assistant.
  • Confidence scores are not yet calibrated between FAQs and knowledge base articles. We suggest using only one of them at a time, as the best result may not always score the highest.