When a Historian Beats a Database
Time-series data has an access pattern that ordinary databases handle badly and specialist ones handle well. The decision usually turns on tag count, query shape and who has to maintain it.
Somebody always asks why the plant needs a historian when there is already a perfectly good SQL server. It is a fair question, and the answer is not that historians are magic. It is that industrial time-series data has an unusual shape, and software written for that shape does a few things very much better.
Consider what the data actually looks like. Ten thousand tags, each producing a timestamped value every second, written continuously and never updated afterwards. Reads are almost always a range over time for a handful of tags — show me these four temperatures for last Tuesday — or an aggregate, such as the hourly average of a flow for the past year. Nobody joins across tags. Nobody deletes a row from the middle.
A general-purpose relational database can store this. It stores it as rows in a table with a tag identifier, a timestamp and a value, and it grows an index. At a few hundred tags this is completely fine and anyone telling you otherwise is selling something. At ten thousand tags it becomes expensive: the index is large, the row overhead is significant relative to eight bytes of data, and a query spanning a year touches an enormous number of pages.
What a historian does differently is mostly compression and layout. Values for one tag are stored together in time order, so a range query reads contiguous blocks instead of scanning an index. Many historians also apply deadband or swinging-door compression, storing a point only when the signal deviates from a predicted line by more than a set tolerance, which for typical process data discards most samples while reproducing the trace within a stated error. The storage difference between that and a naive row per sample is often more than an order of magnitude, and the query difference on long ranges is larger still.
The compression is also the thing to be careful about. Deadband settings are a lossy choice made once and applied forever, and a tolerance chosen for trending will quietly destroy the detail needed later for an event investigation or a control loop analysis. It is worth deciding deliberately which tags are compressed and how hard, and keeping the critical handful raw.
Against that, a historian brings costs that are easy to underestimate. Licensing is frequently per tag, so the architecture that seemed cheap at commissioning gets expensive when someone adds a unit. The query language is usually proprietary, which limits which tools can reach the data, and the export path out of it deserves testing before you depend on it rather than after. And the people who know how to administer it are rarer than people who know SQL, which matters when the person who set it up leaves.
The middle ground has become genuinely good, and it deserves mention because the choice is no longer binary. Time-series databases built on open foundations — the various open-source engines and the time-series extensions to conventional databases — give much of the storage and query behaviour of a historian while speaking SQL and running on ordinary infrastructure. For a site that has a competent IT function and a few thousand tags, that is frequently the right answer.
A rough rule: below a thousand tags with simple trending needs, a conventional database managed by people who already manage databases is fine and cheaper. Above ten thousand tags with long retention and heavy trending, a historian earns its cost. In between, the deciding factor is rarely the technology — it is whether the plant has anyone who will look after the thing in five years, and which of the two they would rather look after.