Share via

FAST ESP 5.x - Indexer and Large Addresses Awareness

  • Article

Hi Search Enthusiasts

I wanted to share an important update that has been made in indexer for Win32 platform.

As you might know, 32-bit process don't address more than 2 GB of Memory by default on Windows platform. To allow them to go all the way to 4GB (usually when hosted on a 64-bit OS), you need to configure the /LARGEADDRESSAWARE flag at linkage time.

See https://msdn.microsoft.com/en-us/library/wz223b1z(v=vs.110).aspx 

You can see that flag in any binary, by using the Dumpbin utility (https://support.microsoft.com/kb/177429) and the /HEADERS switch.

Check for the presence of the following message : "Application can handle large (>2GB) addresses" in the FILE HEADERS section.

 

So far in ESP 5.x, only the fixmlindex and fsearch processes had that flag set. With the release of the latest search engine patch aka esp.5.3.SP5.searchengine.patch02.Win32 , the indexer.exe is now able to address 4GB of Memory.

This improvement should mitigate errors like "Indexer RTSearch: Component is out of memory".

Stay tuned.

Comments

  • Anonymous
    November 23, 2015
    The comment has been removed

  • Anonymous
    December 14, 2015
    @Noori : search engine patch 02 as well? If you have patch02, indexer would be able to address 4GB given that your server has enough overall memory to accommodate. So first I'd make sure the server has enough memory capacity. Memory fragmentation can play a role here as well as the indexer intends to reserve a whole chunk of memory at once. If you have enough mem, look at the oldest data fixml Set directory and the latest Set directory. The difference could cause the indexer to reserve unnecessary memory. Fixml Defragmentation in that case would help. let me know.

  • Anonymous
    March 02, 2016
    Sorry for the delayed response. I'm sorry i was not on the right patch when i posted my comment. I have now applied the recent patch 03 for the search engine 10 days back and it was all going good until yesterday when i observed one of my indexer shutting down. This is still much better than seeing it going down twice everyday during the compaction time. As for the defragmentation, we do it once every month as part of maintenance. But our defrag for a particular SharePoint collection has been failing due to malformed fixmls as a result of which the collections below it have not been going through the defragmentation process since last couple of months. Do you recommend any approach other than using fixmlfeeder to remove the malformed fixmls?