Share via

Add a public website as a knowledge source

  • Article

Public websites as knowledge sources at the agent level use publicly available information to improve the reach of your agent.

URL considerations

The URL used in your agent represents the scope of content for generating responses. There are requirements and restrictions on some URLs.

URL type and structure

  • The URL can have up to two levels of depth—subpaths indicated by a forward slash /. A trailing forward slash, however, is allowed.

    Valid Not valid
    www.contoso.com
    www.fabrikam.com/engines/rotary
    www.fabrikam.com/engines/rotary/    		 www.fabrikam.com/engines/rotary/dual-shaft

  • If the URL redirects to another top-level site, the content isn't included in results:

    For example, if www.fabrikam.com redirects to www.contoso.fabrikam.com, your agent doesn't generate responses from content on either of those URLs.

  • URLs that point to a website, requiring authentication or ones not indexed by Bing.

    For example, wikis and SharePoint sites require authentication, therefore can't be used:

    • fabrikam.visualstudio.com/project/_wiki
    • fabrikam.sharepoint.com

URL domain structure

Any publicly viewable content in the URL you specify, including subdomains under a top-level domain, generate content for your agent.

Examples:

  • If you use www.fabrikam.com/engines/rotary, the content on www.fabrikam.com/engines/rotary/dual-shaft is also used by the agent to generate responses.

    Content on www.fabrikam.com/tools isn't used, since tools isn't a subdomain of rotary.

  • If you use www.fabrikam.com (the www exists), the content on news.fabrikam.com (the www doesn't exist) isn't used, since news. is a subdomain under the top-level domain fabrikam.com.

  • If you use fabrikam.com, then content on www.fabrikam.com and news.fabrikam.com is used, since they sit under the top-level domain fabrikam.com.

Social networks and forum URLs

Your agent might generate nonsensical, irrelevant, or inappropriate answers if you use a forum or social network site as your URL. Therefore, community content on social networks often increases the risk of more answers being rejected.

For more information, see the FAQ for generative answers. AI is trained to avoid generating malicious and offensive responses.

Search engine URLs

Don't include URLs of search engines like bing.com, as they don't provide useful responses.

Add a public website as a knowledge source

To add a public website as a knowledge source, perform the following steps:

  1. Open the agent.

  2. Select Add knowledge from either the Overview or Knowledge pages, or the Properties of a generative answers node.

  3. Select Public websites.

  4. Provide the URL to the public website.

  5. Add a name (by default, the website's URL is used).

  6. Add a description. The description should be as detailed as possible, especially if generative AI is enabled, as the description aids AI orchestration.

  7. Select Add to finish adding the knowledge source.

  8. Test your agent to see how well it responds to questions related to the content from your website. You might want to test edge case questions to decide if you need a lower moderation to be more inclusive.

Note