Geo SCADA Expert

The information available in ClearSCADA is stored in different places called streams. This allows separation of data on disk, the main benefit is for performance reasons, however different streams are structurally different for the specific information they store. The different streams are listed below:

Name Number of Streams per Server Instance Examples Notes
Config 1 for ClearSCADA 2009 R1 and earlier.
Multiple for ClearSCADA 2009 R2 and later, 1 per 16384 object Ids.
Information that doesn't change often:
  • Point limits and anything shown when you click Properties.
  • Logic programs code
  • Mimic content
  • Data grid structure and content
  • Data table structure (not content)
  • Stored in ClearSCADA.Dat, ClearSCADAConfig.Dat or ClearSCADAConfig0000.dat (and ClearSCADAConfigStr.dat) depending on the version
  • Synchronised between servers
  • Flushed to disk only when config has changed (every minute)
  • Saving can have a noticeable performance hit on large databases
  • Is required to run the database
  • Memory resident
Data 1 for ClearSCADA 2009 R1 and earlier.
Multiple for ClearSCADA 2009 R2 and later, 1 per 16384 object Ids.
Information that changes frequently:
  • A point's CurrentValue and CurrentTime
  • A variable's value
  • Stored in ClearSCADAData.Dat or ClearSCADAData0000.Dat depending on the version
  • Synchronised between servers
  • Flushed to disk only when data has changed (every minute)
  • Usually partnered with the config stream, however this stream is not strictly necessary to successfully start a database
  • Memory resident
Dynamic 1 Runtime information that doesn't need to be retained on startup:
  • Outstation's current request
  • Channel's current state
  • Synchronised between servers
  • Not stored on disk, this means on a cold restart all information is discarded
Metadata 1 Extra properties added to objects
  • ClearSCADAMetadata.dat
  • XML file, edited via ViewX (Edit Metadata off root group)
  • Flushed to disk on change
  • Synchronised between servers
  • New in ClearSCADA 2009 R1
  • Lack of file on database start may cause some configuration to be lost
  • Memory resident
Archive 1 Archive volume index.
  • Stored in ClearSCADAArchive.dat
  • Not synchronised between servers (each server has its own local index)
  • Saved following each archive operation
  • Single-purpose stream
Files Multiple, 1 per 'CHisFiles' aggregate
  • Historical Crystal Reports
  • Archived configuration permits
  • EMS safety reports and switching plans
  • MGX configuration upload reports
  • TMX configuration mismatch reports
  • Synchronised between servers
  • Stored in Files directory
  • Disk resident
Historic Multiple, 1 per 'CHistory' aggregate Point historic data
  • Synchronised between servers
  • New records saved to disk every minute
  • Stored in History directory
  • Single-purpose stream
  • 32 bytes per record
  • Disk resident, with a small memory cache
Journal Multiple, many objects to one stream. Object event data and messages
  • Synchronised between servers
  • New records saved to disk every minute
  • Number of streams = (current maximum object id in database/stream size)
  • Stream size can be tweaked for bigger systems.
  • Stored in Journal directory
  • Single-purpose stream
  • 768 bytes per record
  • Disk resident, with a small memory cache
DataFiles Multiple, 1 per type of file for each relevant object
  • Contents of data tables
  • Crystal Report definition (new in ClearSCADA 2009 R1)
  • Latest Crystal Report (new in ClearSCADA 2009 R1)
  • Synchronised between servers
  • Stored in DataFiles directory
  • New in ClearSCADA 2007 R1
  • Disk resident, some objects provide option to cache in memory
ConfigChanges Multiple, many objects to one stream Changes to configuration properties stored historically
  • Disabled by default
  • Synchronised between servers
  • New in ClearSCADA 2010 R1
  • New records saved to disk every minute
  • Number of streams = (current maximum object id in database/stream size)
  • Stream size can be tweaked for bigger systems.
  • Single-purpose stream
  • 1280 bytes per record
  • Disk resident, with a small memory cache
AlarmSummary Multiple, many objects to one stream.
Summarized alarm state changes
  • Disabled by default
  • Synchronised between servers
  • New in ClearSCADA 2015 R1
  • Old records saved to disk every minute (saved after full transition ON-OFF and new ON)
  • Number of streams = (current maximum object id in database/stream size)
  • Stream size can be tweaked for bigger systems.
  • Stored in AlarmSummary directory
  • Single-purpose stream
  • 3304 bytes per record
  • Disk resident, with a small memory cache

The database schema can be used to determine if a property is either a Data or Config stream property, however as a rule of thumb if the property is configurable in the Properties dialogue of an object that property is stored in the Config stream.

The performance impact of the Config stream is that the Config stream is bigger than the Data stream, as such it takes longer to save and requires a longer duration of a lock on the ClearSCADA database. The Data stream is kept small to allow quicker saving, whereas the Config stream has "the rest of the information", usually resulting in the Config stream being approximately five times the size of the Data stream. Because of this impact, configuration changes should be kept to a minimum when using automated tools to change them (i.e. logic and external programs), these programs can change the config often rendering the performance improvement usually seen as minimal. Rapidly changing data should be stored in data properties instead of config properties.

Data grids are also stored in the Config stream and so they are not good as a store for constantly changing data as a change requires a save of the entire config stream for that object, whereas data tables are stored in the DataFiles stream and are more efficient in their storage and transfer.