Indexing Process Fails

Indexing Process Fails

Question

An index request was made for a folder that has successfully been indexed for months prior to this latest error. Examining the indexing results in the Digital Hive Control Center, this error appears.



How can I resolve this issue so that the indexing completes successfully?

Answer

This error can be caused by not having enough free disk space on the Digital Hive server. When Elasticsearch, the search engine within Digital Hive, starts an indexing process, at least 20% of the total disk volume must be available. For example, if the total disk space is 100GB, then at least 20GB must be free and available to the system.

To confirm that this is the issue, open the theia.yyyy-yy-dd.log file located in the <install_location>/app/node1/tomcat/logs directory. Search for the term watermark and an error like this should be present:

blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block]

If this error is present, free up some disk space on the Digital Hive server. Two file directories that could be consuming excessive disk space are:
  1.  <install_location>/app/node1/tomcat/logs
  2. <install_location>/ContentStore/postgresql-backups
It is safe to delete all the log files previous to the current date in the logs directory, unless there is an internal need to maintain more than the current day of log files. The Postgres backups can also be deleted, but the recommendation is to maintain at least a couple of backup files in case they are needed.

    • Related Articles

    • What's New in the 2024.2 Digital Hive Release

      This articles details the new product features, enhancements, and resolved issues, that were included as part of the Digital Hive 2024.2 release. New Features & Enhancements Control Center - Search tab Search is a very powerful use case within ...

    • Content has Disappeared from the Digital Hive Interface

      Question It appears that all of the content has disappeared from the Digital Hive interface. When an Administrator accesses the Explore Content tool from their avatar, a blank screen is returned. How do we restore the view of the content? Answer This ...

    • Connecting to Microsoft PowerBI

      Question How do we connect Digital Hive to Microsoft PowerBI? Answer As a Digital Hive administrator, connecting to Microsoft PowerBI can be accomplished via the following steps: Connecting Digital Hive to PowerBI requires that an app registration ...

    • Connecting Digital Hive to Microsoft Engage (formerly Yammer)

      Question How do we connect Digital Hive to Microsoft Engage so that discussion threads can be embedded onto a Hive page? Answer Connecting Digital Hive to Engage requires that an app registration has been created within Azure. For more information ...

    • Configuring the Digital Hive Health Check application

      Overview Digital Hive introduced an automated way to perform various system checks to ensure that the availability of content, and stability of the system, are performing as expected. The Health Check ability provides administrators with an ability ...