High-speed search and full analytics for Lustre FS
Millions of inodes processed per second

A high-performance Lustre search and analytics utility for demanding HPC environments. Scalable, flexible, and ready for reporting, automation, and modern AI-assisted data workflows.

CONTACT send

Product Goal

Detective Tool provides a comprehensive analytical view of data across your file system and acts as one of the fastest search engines for Lustre.

It helps identify abnormal behavior patterns, performance bottlenecks, and anomalies at early stages, enabling rapid response, preventing failures, and avoiding cluster downtime and financial losses.

Enable fast and convenient file and directory search for administrators on the server. Implement the same capability for end users on the client side in Q3 2026.

"Respond to issues before they turn into incidents."

Details

Detective is a full-featured utility for extremely fast file search and analytics collection in the Lustre file system on MDT and OST. It processes millions of inodes per second and scales to clusters of any size. Over 45 filters are supported (FID, UID, Project ID, size, OST, mtime, and others), enabling flexible and precise searches.

speed

Performance

Millions of inodes per second.

description

Reports

Customized task reports.

hub

Scalability

No cluster size limits.

output

Export

Modern integration formats.

tune

Flexible Search

50+ Lustre filters.

memory

AI / LLM-ready

JSON output for agents.

analytics

Analytics

Full FS visibility.

schedule

Automation

Scheduled execution.

A complete list of capabilities can be provided upon request

Analytics

Detective performs deep analytics of the Lustre file system, using object size as the key metric. The tool builds analytics based on object sizes across files, users, groups, projects, and links; by time attributes, placement parameters, compression information and compression ratio, as well as the directory tree - up to a specified depth.

The analytics is complemented by ranking views of users, groups, projects, jobsID and directories by file count and total data size, enabling fast identification of primary sources of storage growth and filesystem load.

Analytical reports can be output in a human-readable console format, written to JSON and Parquet files, or exported to a dedicated database for further analytics.

Examples

Reports

Detective provides a dedicated reporting layer based on aggregated analytics data stored in the database. Reports are built on summarized metrics, not on per-file records, and focus on changes over defined time ranges – such as days, weeks, or months – capturing filesystem activity and evolution over the selected period.

Because reports rely on global, aggregated analytics rather than individual file data, they require negligible storage space while still providing clear visibility into growth dynamics, usage patterns, and structural changes across the Lustre filesystem.

AI / LLM

If AI systems are used in your production environment, Detective can generate output for both analytics and search results in JSON, which is the optimized output format for AI agents.

By producing compact, structured, and AI-optimized reports, Detective helps minimize processing overhead and significantly reduce AI-related costs while preserving essential context.

Automation and scheduled execution

Detective provides a flexible automation layer built around two complementary mechanisms.

bolt

Event-Driven

Detective can trigger external actions based on search or analysis results. When a condition is met, the tool can execute an external command – for example, automatically removing matched files or invoking custom workflows via external scripts or commands.

update

Scheduled Tasks

Detective supports fully automated execution on a schedule. Scans and data collection tasks can be run periodically – daily, weekly, monthly, or at custom intervals – enabling continuous analytics and reporting without manual intervention.

Together, these capabilities allow Detective to operate as part of an automated infrastructure workflow, not just as an on-demand analysis tool.

Security and open information

security

Security by Design

At HPC-scanners, security is a core requirement for all our products. Detective is designed for safe operation in production HPC environments and does not cause cluster instability or outages. The tool operates in strict read-only mode and never modifies any objects on MDT or OST devices.

settings_input_component

Controlled Resource Usage

To avoid resource contention and starvation, Detective includes an internal resource controller that carefully manages CPU, memory, and I/O usage, ensuring predictable behavior alongside production workloads.

Transparent Core

We also recognize the importance of transparency for security validation. For this reason, we provide limited access to the scanning engine source code under the "Detective – Community Edition License". This allows users to review and verify the safety of the core scanning logic. GitHub open_in_new

Business Benefits

manage_search

Faster data search and analysis

visibility

Greater visibility into file system usage

savings

Lower storage and infrastructure maintenance costs

query_stats

More accurate budgeting and cost forecasting

gpp_good

Reduced risk of incidents

storage

More efficient use of storage resources

swap_vert

Better management of hot and cold data

auto_mode

Less manual routine work

License benefits

cycle

Regular Updates

New Features Every Quarter

full_stacked_bar_chart

Full Access to Capabilities

All Features Without Limitations

verified

Support for Current Versions (LTS)

Stability and Support with an Active License

flex_direction

Flexible Licensing Terms

For Various Scales and Scenarios

speed

Fast Critical Fixes

Fixes in the Shortest Time

policy

Security and Change Control

All Updates Undergo Internal Validation

engineering

Expert Support

Direct Work with Engineers

history

Support for Legacy Versions

Adaptation to Current Cluster Capabilities

lightbulb

Custom Functionality

You Propose Ideas — We Implement Them

branding_watermark

Branding (OEM)

Available with an OEM License

Contact

“Control today - stability and savings tomorrow.”

Frequently Asked Questions

How fast is scanning? expand_more

When a PCIe NVMe drive is used for the MDT, scanning one million inodes typically takes less than 1 second. OST scans are generally slower. Actual performance depends on the number of filters applied and the amount of metadata collected.

What is the difference between analytics and reports? expand_more

Analytics shows the state of the filesystem at the moment of a scan. Reports are generated from accumulated analytics over a specified period.

Can Detective output be integrated with Grafana or other systems? expand_more

Yes. For example, Grafana can consume data exported in JSON format.

Which Lustre versions are supported? expand_more

The current release is aligned with the latest Lustre releases and the master branch. Older versions are also supported, although some limitations may apply.

Can Detective scan multiple devices at once? expand_more

Yes. Detective can scan both OSTs and MDTs, but only sequentially. This approach is intentional and helps reduce load on the MDS while preserving operational safety.

Can Detective reconstruct the directory tree without a mounted client? expand_more

Yes. Detective stores the FID/Name directory tree in its local database, enabling directory reconstruction without a mounted client.

Does Detective collect changelog-based analytics? expand_more

Yes, but only aggregated event counters are stored. No information about individual files is retained.

Can Detective run on a schedule without user involvement? expand_more

Yes. Scheduled operation can be configured through the configuration file. Detective can also run as a service when that mode is enabled.

Can Detective export analytics to external databases? expand_more

Not directly. Detective currently supports export in Parquet and JSON formats. Support for an optional per-file database is planned.

Which output formats are supported? expand_more

Supported output formats include text, JSON, Parquet, and SQL.

Does scanning impact filesystem performance? expand_more

Not directly. However, scanning does consume MDS resources, including CPU and memory. Built-in throttling controls can be enabled when required.

How large is the local Detective database? expand_more

Database size depends on the volume of stored data, but in most environments it remains below 6 GB.

Is a trial version available? expand_more

No. Detective is available only as a fully licensed product.

How is licensing structured? expand_more

Licensing is typically provided under a 1-year agreement with automatic renewal options. OEM licensing and branding options are also available.

How are updates delivered? expand_more

Customers receive update notifications, and access to new versions is provided through the cloud in accordance with the applicable license terms.

What is included in quarterly releases? expand_more

Quarterly releases generally include new functionality, non-critical updates, and ongoing improvements.

How is support handled? expand_more

Issues can be reported by email. An incident ticket is automatically created in HPC-SCANNERS. Incidents are typically infrequent and are usually resolved within a short timeframe.

Are vendor-specific Lustre distributions supported (DDN, HPE, TLC, etc.)? expand_more

In most cases, yes, as vendor distributions are usually based on public Lustre FS versions. However, certain vendor-specific business features may not be supported.

Still have questions? Feel free to ask by email or