ClearSCADA allows you to optimize the historic granules for historic data, events and configuration changes, (that fall within the Manage the Size of the Primary Index). A Historic Types and Streams is a file that holds the historic data for one item for a specific time period - 1 week for historic data, 1 hour for events and configuration changes. The optimization process helps improve performance and reduce memory and disk usage when querying historic granules by:
- Sorting the historic records into chronological order. This means ClearSCADA will not have to sort the records when loading from disk.
- Removing any corrupt records.
- Removing any deleted records. When records are deleted, they are marked as deleted but are not removed from the granule. Consequently, they continue to use disk space and memory.
- Removing any duplicate records.
To optimize events and configuration changes, you should use the compression feature (optimization occurs during compression for these types of historic). For more information, see Compressing a Database.
To optimize historic data, use the Start Historic Optimization method. Start Historic Optimization is available for the CHistorian class of the Root Group (which you can view by using the Database Schema). The Start Historic Optimization method requires the System Admin permission and can be invoked in the same way as other methods (Mimic animation, Logic program, Schedule, Method Call, Alarm Redirection, and so on). There is no built-in pick action for the Start Historic Optimization method.
There is also an Optimize Data method that is available for the CHistoryBase class. The Optimize Data method allows you to optimize the historic granules for a single database aggregate (whereas the Start Historic Optimization optimizes the historic granules for the entire database).
Typically, you will only need to optimize your historic data once. For this reason, you should invoke the Start Historic Optimization method at a time where the historic data for the relevant time period is unlikely to be modified. If your system uses the long term storage index(see Manage the Size of the Primary Index)you will need to consider the time limits on the primary index and the secondary index, (see Specify the Duration to Retain Historic Data Online).
ClearSCADAcan only optimize historic data granules while they are writable (still in the primary index and not yet available for archiving).
When you invoke the Start Historic Optimization method, you may be prompted to enter a time range (unless a time range has already been set as an argument for the method). The amount of time you specify is the time span that will be optimized in the historic records - those records that fall outside of the defined time span will not be optimized. The amount of time it takes to complete the optimization process will vary depending on the number of records in the defined time range - the more records there are, the longer the process will take.
Historic optimization for Events and Configuration Changes is always applied to the entire writable time period.
Once invoked, optimization takes place as an asynchronous background task and does not affect the operational availability of your system. When optimization is complete, an event is logged, detailing the optimization changes that have been made. You can view the progress of the optimization process by viewing the Events List for the Root Group. The Events List logs the start of the optimization process, what action the system took to optimize the data and the end of the optimization process.After optimization, historic queries on your system will be more efficient. The performance, memory usage and disk usage benefits will be most noticeable on systems where:
- records are not in chronological order
- large amounts of historic data are deleted.
Example:
The Start Historic Optimization method can be configured to be performed upon user selection on a Mimic or according to a schedule.
Configuring a Mimic
To configure a Mimic so that it provides access to the Start Historic Optimization method:
- In ViewX, display a Mimic in Design mode or create a new Mimic and display it in Design mode.
- Create the Mimic object that is to act as the 'trigger' for the Start Historic Optimization method. When this object is selected (in Run mode), the system will optimize historic data for use.
- Use the Pick Action Wizard to associate the Mimic object with a Method pick action. You need to define these settings:
- Type of Pick Action: Method
- Object: The Root Group of your system has to be the Object. The Start Historic Optimization method is only available to the Root Group, for example, SCX:////CRootGroup/$Root
- Relative: Disabled
- Aggregate: Historian
- Method: Start Historic Optimization
- Arguments: None defined
- Pick Action Comment: None defined
- Proceed through the steps of the Pick Action Wizard and select the Finish button when the configuration is complete.
- Save the Mimic.
When the Mimic is displayed in Run mode, the Mimic object that you added in step 2 will trigger the Start Historic Optimization method. The user will be prompted to enter a start time and an end time—these times define the range of historic values that will be optimized. The default settings are usually suitable.A copy of the button is added to the Mimic. The copy is reconfigured to be associated with the Cancel Historic Optimization method. This will allow a user to abort an optimization process that is currently underway.
A user with System Admin permissions accesses the Mimic in Run mode and selects the Optimize Historic button. A prompt is displayed, requesting the user to enter a time range. (This is because the time range was not specified as arguments in the pick action configuration).
The user enters a Start Time of WEEK - 10W and End Time of WEEK + 1W and then selects OK.
The historic optimization process begins as a background task and will optimize the historic records from 10 weeks ago up to the end of the current week (WEEK is the start of the current week).
Shortly after the optimization process has begun, the user realizes that it would be more practical to run the optimization process overnight. The user selects the Cancel Optimization button on the Mimic to abort the optimization process. Those granules that have already been optimized as part of the process will remain optimized. The other granules will be optimized at a later time.
If you want to configure a Mimic to indicate that the historic optimization process is in place, you can animate a Mimic object so that it is associated with the OptimiseInProgress tag of the Root Group. The OptimiseInProcess tag is within the Root Group's Historian tags.
The OptimiseInProgress tag stores a Boolean value. When the historic optimization process is in progress, the tag stores a True value; when it is not in progress, the tag stores a False value.
Configuring a Schedule
To configure a schedule to trigger the Start Historic Optimization method at regular intervals:
- In ViewX, create a new Schedule.
- Add an Action on the Schedule. The Action has to have these settings:
- Object—The Root Group of your system has to be the Object. The Start Historic Optimization method is only available to the Root Group.
- Aggregate—Historian
- Method—Start Historic Optimization.
- On the Schedule Editor, double-click on the times at which the Start Historic Optimization method is to be triggered.
- Save the Schedule.
Further Information
Configuring Mimic Animations: see Animations in the ClearSCADA Guide to Mimics.
Schedules: see Using Schedules to Automate Regular Functions in the ClearSCADA Guide to Core Configuration.