Why is my incremental crawl not stopping on SharePoint Server 2016 on premise?

Question

I've had a SharePoint Server 2016 search crawl set up for about a 18months now. We have recently had issues with the search incremental crawl. I've restarted servers, stopped the crawl, restarted it. When I restart the search it kicks off a full crawl, this completes, then the incremental crawl starts at a scheduled time and should take around 3 minutes but it is on 40 hours now and continues.
There seems to be quite a few security updates being triggered? Before the issues it was a consistent 417 but now it is over 1 million...

The search application is split between four servers and all components are working with green ticks.

Any help is really appreciated.

Thanks.

Answer 1

Hi @Dominic McConville ,

There can be several reasons why your incremental crawl is not stopping on SharePoint Server 2016. Here are a few considerations:

1. Crawl Errors: If there are items that are returning errors during the crawl, it could cause the incremental crawl to hang. Continuous crawls do not process or retry items that repeatedly return errors, which might lead to a backlog that affects the incremental crawl.
2. Crawl Load: Since you mentioned that the search application is split between four servers, ensure that the load is properly managed. High load on the crawler or crawl targets can lead to delays in processing.
3. Security Updates: The increase in security updates being triggered could indicate that there are many items that need to be re-evaluated for security permissions. This can significantly increase the time taken for the incremental crawl.
4. Full Crawl Behavior: When you restart the search service, it kicks off a full crawl, which is expected behavior. However, if the incremental crawl is taking much longer than usual, it may indicate underlying issues that need to be addressed.

If the problem persists, consider reviewing the crawl logs for any specific errors or warnings that might provide more insight into the issue.

References:

• Best practices for crawling in SharePoint Server
• Manage continuous crawls in SharePoint Server
• Start, pause, resume, or stop a crawl in SharePoint Server

---

If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.