Define the Historic Group Size to Optimize Performance

The ClearSCADA historian stores the historic data in a series of files on disk. For each historic item, there is one file for each weeks' worth of data, and these files are organized in a hierarchical folder structure. The depth and breadth of the hierarchy can affect the performance of the historian, as for each query, the Windows file system has to locate the relevant files in the folder structure.

To change the Group Size setting for Historic Data:

1. Access the ClearSCADA Server Configuration Tool (see Accessing the ClearSCADA Server Configuration Tool).
2. Expand the Historic Configuration branch.
3. Select Historic Data.

   

4. Use the Group Size combo box to choose the appropriate size for each group.

   The Group Size setting defines the maximum number of <Items> sub-folders permitted for each group folder. For example, with a Group Size of 8 there is a maximum number of 256 <Items> sub-folders, whereas a Group Size of 10 sets the maximum number to 1024 sub-folders. The greater the number of sub-folders in a folder, the longer it will take the filing system to access a file (in one of the sub-folders). By default, the Group Size is 8 (256 Objects), but you can increase or decrease this amount as required.

   When the Group Size is zero (no groups) the following hierarchy is used:

   <Base>\<Aggregates>\<Items>\<Granules>

   When the Group Size is non-zero the following hierarchy is used:

   <Base>\<Aggregates>\<Groups>\<Items>\<Granules>

   Where:

   • <Base> is the location of the historic data (see Locations of the Database Folders)
   • <Aggregates> are the folders for each historic aggregate. The name of each folder is the name of an aggregate (typically, there is just one aggregate named 'Historic').
   • <Groups> are the optional group folders, which divide the item folders into group sub-folders. The name of the folder indicates the group size and group number.
   • <Items> are the folders for each historic database item. There is one folder for each database item that has historic enabled. The name of the folder is the item's ID (padded to six digits).
   • <Granules> are the files containing the actual historic data, with one file for each week. The name of the file indicates the week to which it relates, and the file extension indicates the historic data type (see Historic Types and Streams).

   The following table indicates the average number of files Windows will iterate through for each Group Size setting. The table shows the average number of iterations for the Group Sizes of 0, 4, 6, 8, 9, and 10 for 100, 1000, 5000, 25000, 100000, and 250000 aggregates. You can use this as a guideline for your own system. But for maximum efficiency, you should calculate the average for your system (use the specific number of historic items on your system).

   Consider the number of historic items your system currently has, and is likely to have in the future. You can then choose a Group Size that will result in a low number of iterations (the table illustrates some useful examples). These values will provide the most efficient historian performance.

   	0 - No Groups	4 - 16 Objects	6 - 64 Objects	8 - 256 Objects	9 - 512 Objects	10 - 1024 Objects
   100 Items	50	12	33	129	257	513
   1000 Items	500	40	40	130	257	513
   5000 Items	2500	165	72	138	261	515
   25000 Items	12500	790	228	177	281	525
   100000 Items	50000	3133	846	324	354	561
   250000 Items	125000	7821	1926	617	501	635

5. Right-click on the System icon in the tree-structure, and then select the Apply Changes option to update the server's configuration. The change will come into effect the next time the server is restarted.

   ATTENTION: Changing the Group Size means the existing folders have to be rearranged. If there are a large number of folders, this rearrangement may take a considerable amount of time. As a result, the server may take a long time to start-up following a change in Group Size.