SAP HANA architecture

HANA stores data primarily in memory (persisted to disk for durability) and, for most application data, in a column store. In-memory access removes disk-I/O latency, and columnar storage makes scans and aggregations extremely fast and highly compressible, so analytics can run directly on live transactional data.

• Column store: default for application/analytical data; fast scans, aggregation, compression.
• Row store: for a few technical/frequently-single-row-accessed tables.

• Index server: the main engine (SQL, calculation, stores).
• Persistence layer: data volumes + log volumes for durability and recovery.
• Multitenant database containers (MDC): a system DB plus isolated tenant DBs.

The phrase in-memory worries people, and the persistence design is worth understanding because it is the answer to the obvious question.

Data lives in memory for speed, and it is written to disk in two forms. Savepoints periodically write the changed pages to the data volume. The redo log is written synchronously at commit to the log volume, so a committed transaction is durable the moment it commits.

After a crash, HANA reloads from the last savepoint and replays the log from that point forward. Nothing committed is lost. The restart takes as long as loading the data back into memory, which on a large system is the reason startup is not instant.

This is also why the two volumes have different requirements. The log volume needs low latency because every commit waits for it. The data volume needs throughput for savepoints and for reload.

Delta merge is the other mechanism worth knowing. Writes go to a small write-optimised delta store rather than into the compressed columnar main store, and a periodic merge folds them in. A table whose merge is not happening grows a large delta and slows down, which is one of the more common performance findings. See HANA backup for the recovery side.

By collapsing the old transactional/analytical split, HANA let SAP remove redundant aggregate and index tables, simplify the data model (e.g. the universal journal), and deliver real-time analytics, the foundation of S/4HANA.

Concretely, it removed the need for a great deal of pre-aggregation. Totals tables existed because summing millions of line items on demand was impossible; on HANA it is fast enough to do on the fly. That is why S/4HANA could collapse the finance tables into a single journal and drop aggregate tables, and it is the mechanism behind most of the simplifications. See calculation views for the modelling layer built on it.

• Sizing memory. The working data set has to fit, plus room for temporary results and growth. Undersizing produces unloads and unpredictable performance rather than a clean failure.
• Scale up or scale out. One large machine is simpler and is the normal answer for transactional systems. Scale-out spreads tables across nodes and suits very large analytical workloads, at the cost of partitioning decisions.
• High availability. System replication keeps a second system in step, and the sync mode chosen decides whether a commit waits for it.
• Data tiering. Not all data needs to be in memory. Warm and cold tiers move older data to cheaper storage, and deciding what qualifies is a business question about how far back people actually query.

• Where the log volume lives. Every commit waits for it, so its latency is felt by every user of the system, and putting it on the same storage as the data volume is a false economy.

Architecture on paper meets reality in how the system is actually run, and a few choices define that.

Appliance or tailored datacenter integration. An appliance is a certified, preconfigured hardware and software bundle: fast to stand up and inflexible. TDI lets you use your own certified components, which most organisations prefer because it fits existing storage and networking. Cloud deployments are a third route where the provider handles the underlying certification.

System replication is the standard high availability mechanism. A secondary system receives the redo log continuously and can take over. The replication mode is the decision that matters: synchronous means a commit waits for the secondary to confirm, which costs latency and loses nothing on failover; asynchronous does not wait, which is faster and can lose the last transactions.

Preload decides whether the secondary keeps data in memory ready to serve. With it, takeover is quick; without it, the secondary must load first, and a large system takes a while doing that.

Backups are separate from replication and neither replaces the other. Replication protects against a system failure. Backups protect against a mistake, since a deletion replicates faithfully to the secondary. Both a data backup schedule and log backups are needed, and a recovery that has never been tested is an assumption.

Multitenant database containers allow several isolated tenant databases on one system, sharing resources while keeping data separate, which is how landscapes consolidate without merging systems.

Sizing is where HANA projects most often get an unwelcome surprise, and the components are knowable.

Static data is the compressed column store: the tables themselves. Compression on real business data is substantial, and it varies by table, which is why sizing is done from a real system rather than from a rule of thumb.

Dynamic working memory is what queries need while they run: intermediate results, sorts, joins. A rough convention is to allow as much again as the static data, and analytical workloads can need more.

The delta store holds recent writes before merge, and a system with heavy write activity and infrequent merges carries more here than expected.

The sizing reports SAP provides read an existing system and estimate what it would need, which is far more reliable than estimating from table sizes. For a new implementation there is no such system, so sizing is done from expected transaction volumes and revised once real data exists.

What happens when memory runs short is worth knowing: tables are unloaded from memory and reloaded on next access. The system keeps working and becomes unpredictably slow, which is harder to diagnose than a clean failure. Watching the unload statistics is how that is caught before users report it.

• Assuming HANA is just a faster disk DB, it is architecturally different.
• Ignoring memory sizing, data must fit in RAM.
• Overlooking persistence, in-memory still needs durable storage.

• Ignoring delta merge. A table with a large unmerged delta performs badly and nothing obvious points at the cause.
• Treating memory as unlimited. Selecting everything and filtering afterwards is expensive here in a way it was not on a disk database with a good index.
• Assuming an ECC system on HANA is S/4HANA. See HANA performance and HANA security for the operational detail.

• Sizing from table sizes rather than from a sizing report. Compression varies enormously by table, and estimates made without measuring are usually wrong in the expensive direction.
• Assuming replication is a backup. A deletion replicates perfectly.

Understanding the architecture is the start, and sizing, monitoring and tuning a real system is the part you do in the course.

The idea worth carrying is that in-memory changes what is worth doing, not just how fast it happens. Aggregates that existed because summing was impossible are gone, and that is the mechanism behind most of what changed in S/4HANA.