Indexing Performance
The indexing-performance script benchmarks the indexing performance provided by a data source/historian at specific index resolutions.
Download the support script package from the download server.
Log in using your TrendMiner file server credentials. If you don't know these or your credentials are not working, please contact your TrendMiner Customer Success Manager or send a support request to support@trendminer.com
This script is executable from every installation which has access to the TrendMiner connector which is connected to the data source/historian for which you want to test the indexing performance. There are different scripts for different operating systems included in the downloadable file (Windows, Linux, MacOS).
This article focuses on the Windows version: the indexing-performance.exe script which can be executed on Windows machines.
Fill in the configuration. 3 config files are provided in the downloaded files: config.json, config-batch.json and config-performance.json.
For the indexing performance only the config-performance.json is needed.
Open this file (preferably in notepad++) and adjust it according to the setup. The following info is required to run this script.
{ "Connector": "http://trendminer-connector.example.org/", "StartDate": "2022-01-01T00:00:00Z", "EndDate": "2023-01-01T00:00:00Z", "Resolutions": ["60", "15"], "GranularityDays": 31 }Connector: the URL of the connector which is used to connect to the historian/data source.
StartDate: the start date of the benchmarking run
StartDate: the end date of the benchmarking run
Resolutions: A list of index resolutions to test, in seconds
GranularityDays: the number of days to fetch from the historian in one request. Any number >31 will default to one month.
Prepare a tags.txt file containing all tags which need to be benchmarked.
File format requirements:
No header
One tag name per line
Tag prefix needs to be added!
Example of content of tags.txt:
Run the script
The indexing performance script supports the following parameters:
-config string
Configuration file to use (optional) (default "config.json")
-historianName string
Historian/data source name to benchmark. This name should be the data source name configured in ConfigHub, or you can fetch the list of configured data sources via the connector api: How to get the list of configured data sources?
-tags string
A file containing a list of tags (optional) (default "tags.txt")
-batch
Run without expecting user input (optional)
Make sure the tags.txt file and the config-performance.json file are in the same windows folder as the indexing-performance.exe
Open a command window and run the indexing-performance.exe script with the correct parameters.
Example command:
indexing-performance.exe -tags my-tags.txt -config config-performance.json -historianName "My IP21 data source"
The output looks like:
{"TagName":"SINUSOID","BlockSize":60,"Count":6580,"Timings":[15,16,16,15,15],"Error":""}
{"TagName":"SINUSOID","BlockSize":15,"Count":6580,"Timings":[15,15,15,15,15],"Error":""}In this case the tag SINUSOID was tested at one minute and 15 seconds index resolution. The index resolution determines the number of data points which are requested. Resolution = 60 means the requested time period is divided into blocks of 60 seconds and max 4 points per interval are requested.
The time period between start and end was requested 5 times per tag and the timings are listed under 'Timings' (expressed in seconds).
In this example it took around 15 seconds per run to fetch the tag data. Since the number of data points (Count) was the same for both resolutions, lowering the resolution would not result in more detailed search results in TrendMiner. But lowering the index resolution would also not affect the indexing performance for this particular historian since it took around the same amount of time to fetch the higher resolution data.