When a ClearSCADA server is started it creates and maintains a Primary index of the historic data granules in that server (a granule is a file that holds the historic data for one item for a period of one week, see Historic Types and Streams). The Primary index allows your system to retrieve historic data quickly. The Primary index is created and kept in memory and is rebuilt each time that the server is restarted.
ClearSCADA does not create an index for Alarm Summary, Event Journal or Configuration Changes data that is kept online.
Understanding the Primary index
The size of the Primary index, and the amount of memory it requires, is dependent on:
- The number of items with historic data
- The amount of time for which the historic data is kept online, see Specify the Duration to Retain Historic Data Online.
You can use the following calculation to determine the maximum amount of memory required for the Primary index:
Total Historic Items × Primary Weeks × Primary Granule Size
| 64-bit Server | 32-bit Server | |
|---|---|---|
| Primary Index | 96 bytes per granule | 68 bytes per granule |
Primary index and Keep online for
In this example we are assuming a 64-bit to illustrate the effect of the Keep online for value on the Primary index and how this impacts on the use of the system memory.
- A 100,000 historic items kept online for 1 year would require about 499MB for the primary index.
- A 100,000 historic items kept online for 25 years would require 12GB for the primary index.
As the example above illustrates changing the number of historic items, and / or more importantly the length of time that the historic data is kept online, can increase the memory requirement significantly, which can have a major effect on the performance of the server.
Understanding the Secondary index
ClearSCADA provides a feature that allows you to limit the size of the Primary index. The feature works by defining the amount of online historic data that remains in the Primary index, and ClearSCADA creates a second more compact Secondary index for the remaining online historic data. The Secondary index takes up much less memory per granule than the Primary index, and so the index is smaller as indicated in the table below.
| 64-bit Server | 32-bit Server | |
|---|---|---|
| Primary Index | 96 bytes per granule | 68 bytes per granule |
| Secondary Index | 8 bytes per granule | 8 bytes per granule |
How the Secondary index works with the Primary index
A 64-bit system with 100,000 historic aggregates kept online for 25 years with 1 year in the Primary index and 24 years in the Secondary index will require up to 1.4GB of memory. If the Secondary index feature is not used, the same system would require 12GB for the primary index, (so this feature produces an 88% saving in memory usage).
How to manage the Primary index
The Primary index is by default applied to all historic data. When deciding how many weeks’ worth of historic data to store in the primary index, you should consider the following factors:
- For how long is the historic data being kept online? This is defined by the Keep online for setting (see Specify the Duration to Retain Historic Data Online)
- How much memory is available in the server?
- How far back in time do you need to create new historic records, delete existing records or optimize granules?
- How far back in time do you need to be able to transfer historic data to the server’s partners (if any).
If archiving is also enabled, then the period must be equal to or greater than the archive period (defined by the Archive After setting (see Define the Archive Time for Historic Data).
To manage the size of the Primary index and create a Secondary index:
- Access the ClearSCADA Server Configuration Tool.
- Expand the Historic Configuration branch.
- Select the Historic Data entry.
-
Use the Index After check box to manage the size of the Primary index. (Select to enable the feature; clear to disable the feature).
- Right-click on the server icon and select the Apply Changes option to update the server’s configuration. The changes will come into effect the next time the server is restarted.
If you select the Index after check box and use the field to define how long the historic data granules will be in the Primary index, granules that are older than the number of weeks you specify will be in the Secondary index. The period is measured from the start of the week in which the server was last started.
The higher the number / duration, the more memory will be required for the index.
In this example the Keep Online for is set to 25yrs (1300 weeks) and the Index after is set to 1yr (52 weeks).
Initial state of the Primary and Secondary indexes
When the system is initially started ClearSCADA creates a Primary index of 449MB and a Secondary index of 998MB as illustrated below.
The initial size of the Primary and Secondary index is dependent on the amount of data that exists on the server when it is started.
Operational state of the Primary and Secondary indexes
Unlike the Keep Online for, Archive After and Future Data time periods, the Primary index time is not a sliding window (it does not move forward week-by-week as time goes by). This means that after the initial start of the server new granules are created and added to the Primary index, which will grow in size. The Secondary index will either remain unchanged or decrease in size if old data exceeds the Keep online for period, until the server is restarted. ClearSCADA then rebuilds the indexes and resets them to the initial state.
The example below illustrates what would happen if the server was run continuously for 5 yrs following a restarts.
Although the Index after setting is 1yr (52 weeks) it does not affect the indexes until the server is restarted.
Calculate the Amount of Memory Required by the Primary and Secondary Index
The amount of memory required by the Primary and Secondary index is determined by four factors:
- The total number of historic aggregates. Typically, this is the number of points and accumulators that have historic enabled. However, there are also other historic aggregates.
- The number of weeks covered by the historic data in the primary index. This amount is defined by the Index After setting, (see Manage the Size of the Primary Index).
- The number of weeks covered by the historic data in the secondary index. This amount is equal to the difference between the Keep Online setting and the primary index period (defined by the Index After setting).
- The granule size. This varies between 32-bit and 64-bit servers:
- 68 bytes for 32-bit servers
- 96 bytes for 64-bit servers.
- Use the following calculation to determine the maximum amount of memory required for the Primary and Secondary index:
(Total Historic Items × Primary Weeks × Primary Granule Size) + (Total Historic Items × Secondary Weeks × Secondary Granule Size)
Example:
A 64-bit system has 100,000 historic items that are kept online for 5 years. They spend 1 year (52 weeks) in the primary index and 4 years (209 weeks) in the secondary index.
(100000 × 52 × 96) + (100000 × 209 × 8) = 635.5MB
This means up to 635.5MB memory is required.
How to determine the number of historic aggregates
- Access the ClearSCADA Server Status tool.
- Expand the Historic branch for the required server.
- Select the Historian entry.
- Examine the Total Stream entry for the Raw row. This is the total number of historic items within the Historic aggregate, (see Historic Types and Streams).
You can also use the Total Primary File and Total Secondary File entries to calculate the current (rather than maximum) memory usage. Also, the Total Memory Usage entries indicate the amount of memory used by the Historian, including the granule index.
Effects of Queries and Archive Volumes
When granules in the secondary index are queried, they are promoted into the primary index. Additional memory is required while the granules remain in the loaded granule cache. The size of the cache and the queries being performed will determine how much additional memory is required.
All granules in a mounted archive volume are added to the primary index. Additional memory is required until the archive volume is dismounted.
Using this information you can determine the best compromise between the memory required by the primary index and the restrictions imposed by using this feature.
Secondary Index Restrictions
When the Secondary index is enabled in a Hot Standby / Main server, granules that fall within the Secondary index have certain restrictions:
- The granules are read-only, this means:
- New records cannot be added to the granules. The CHistoryBase:LoadDataValuesEx() method cannot be used for adding records to the granules, see Importing Historic Data.
- Existing records cannot be deleted from the granules.
- Optimization cannot be applied to the granules (see Historic Optimization).
- The granules are not transferred to any partner servers, as this feature effectively reduces the Keep online for window that is synchronized between Hot Standby servers.
However, a permanent standby server is able to receive updates to the granules in its secondary index from its partners. This allows the Secondary index to be used on a permanent standby server when it is not used on its partners. This is true even if there is an overlap between the secondary period on the permanent standby server and the online period on its partners.
Example:
If a granule in the Main server's Primary index is altered, the data is transferred to the other Hot Standby server(s) and the Permanent Standby servers. The transfer occurs even if the granule falls in the time period covered by Permanent Standby servers' secondary indexed period.
When a granule in the secondary index is queried, it is promoted into the primary index (and therefore will require more memory). It is demoted back into the secondary index when it is purged from the loaded granule cache. (For more information on the loaded granule cache, (see Define the Historic Data Cache Size).
Only granules in the Raw and Suppression historic data types support the long term storage index. Granules in the Modified and Annotation historic data types only have a writable / syncable period, (see Historic Types and Streams).
If you are using both archiving together with the Primary and Secondary index, the granules in the secondary index can be archived. When an archive volume is mounted, all of the granules contained on the archive volume are stored in the primary index, regardless of their age.