Data is growing, so does Exadata too…
One of the EMC paper for 'Data Growth, Business Opportunities, and the IT Imperatives' says, Data is growing 40% annually. EMC is now Dell.
Does Exadata also growing? Exadata was first released in 2008 as Oracle's first "Engineered System", popularly known as "hardware and software engineered to work together". Since then we have seen 6 versions of Exadata and most recent is X5 Series which has X5-2 and X5-8 Exadata. Let's talk in brief about history of Exadata, their key components and how they evolved.
Let's see How Exadata evolved during these years and a journey of a better product to the best.
V1, the first Exadata, was released in late 2008. It was known as Exadata V1 and was a
combination of HP hardware and Oracle software. It has no Flash.
V2, the second version of Exadata, was introduced at Open World in 2009. Oracle partnered with Sun. The biggest difference was the addition of a significant amount of solid state-based storage. The storage cells were enhanced with 384G of Exadata Smart Flash Cache. Oracle software was also enhanced to take advantage of the new cache.
X2 was released at Oracle Open World in 2010.First time there were two distinct versions of the X2. X2-2 and X2-8. The X2-2 has eight dual-socket database servers with hex-cores. X2-8 has larger number of CPU cores and has larger memory footprint. In 2011, Oracle changed the hardware in the X2-8 to 8 10-core CPUs and 2TB of memory per node. Storage expansion racks were introduced.
In January 2012, Oracle increased the size of the high-capacity disks from 2TB to 3TB
X3 was introduced in 2012. Compute node updates included eight-core Intel Sandy Bridge CPUs and increased memory up to 256GB per server. Storage servers saw upgrades in CPUs and memory, and flash storage increased to 1.6TB per server. The X3-2 family also introduced a new size—the eighth rack. X3-8 rack had some improvements in the storage servers, but the compute nodes in X3-8 racks were same as their X2-8 counterparts.
X4 version was released in 2013. In this version processing was increased to 2x12 core CPUs. It had ability to upgrade memory to 512GB in a compute node. Flash and disk storage was also increased. It featured new model of high-capacity disk. 600GB disks were retired, and included 1.2TB, 10,000 RPM disks. These disks were a smaller form factor (2.5" vs 3.5"). The other big change with the X4-2 was the introduction of an active/active InfiniBand network connection. On the X4-2, Oracle broke the bonded connection and utilized each InfiniBand port independently. This allowed an increased throughput across the InfiniBand fabric.
X5 was announced in early 2015, sixth generation of Exadata. The X5-2 was a dramatic change in the platform, removing the high-performance Disk option in favor of an all-flash, NVMe (Non-Volatile Memory Express) model. High-capacity disk sizes remain same at 4TB per disk. Size of the flash cards doubled to 6.4TB per storage server. Memory stayed consistent with a base of 256GB, upgradeable to 768GB, and the CPU core count increased to 18 cores per socket. With X5, customers are allowed to purchase X5-2 with any configuration required—a base rack begins with two compute nodes and three storage servers
Let's compare individual Exadata features and associated components
Storage Server CPUs, Disk Space and Flash Cache
Most frequently-accessed data on the spinning disks are automatically cached in the Exadata Smart Flash Cache area on high-performance flash disks. Exadata Smart Flash Cache provides a caching mechanism for frequently-accessed data on each Exadata Cell nodes.
The Exadata Smart Flash Cache area on flash disks is automatically created on Exadata Cells during startup. Regular disk gives a few hundred IO/Sec but Flash can offer millions of I/O s per second. Exadata software transparently moves hot data to flash cache. Exadata full rack system has 14 storage servers with total 89.6 TB Flash cache. This Flash cache can provide 140GB/S throughput or it can manage up to 4.1 Million IOPs on full rack.
Another key beneficiary of this growing flash is Smart Scan / Storage indexes. This is a very powerful feature of Oracle Exadata Storage Server Software. Storage indexes help to avoid I/O operations. Oracle Exadata Storage Server Software creates and maintains a storage index in Exadata memory. The storage index Keeps track of minimum and maximum values of columns for tables stored on that Cell. This functionality is done transparently, and does not require any action. Any smart scan query is sent to storage via iDB (intelligent database) protocol. Once data is offloaded all the SQL processing happens inside Exadata Storage Servers including data filtering and processing. Only the rows and columns that are directly relevant to a query are sent to the database servers. This save a lot of Database server CPUs work.
Exadata Hybrid Columnar Compression (EHCC) is a new compression technology exclusive to Exadata. Oracle explains that you can achieve 10x to 50x compression based on what type of compression you choose and your data.
EHCC is a new method for storing data within a database block. Data within the data block is neither stored row wise sequentially nor column wise. As the name implies, EHCC utilizes a combination of both row and columnar methods for storing data. Because of the hybrid approach to achieve this compression we get the benefits of columnar storage while avoiding the performance drawbacks of a pure columnar format. Hybrid Columnar Compression often delivers 10X-15X compression ratios
Above slide shows flash cache growth pattern with various Exadata releases
Exadata X5-2 introduces Extreme Flash Storage Servers. Each Extreme Flash storage server contains eight 1.6 TB state-of-the-art PCI Flash drives. PCI flash delivers ultra-high performance by placing flash memory directly on the high speed PCI bus rather than having slow disk controllers and directors on moving disks. We get total of 179.2 TB Flash. With This option read performance remains same but write performance increase from 2.6M IOPS to 4.1M IOPS. In this option HC disks are replaced with Flash.
Storage software provide many other great features which also gets benefited by growing CPU and Flash like:
Smart Flash Logging
Columnar Flash Cache
Exhaustion Direct to Wire OLTP Protocol
IO and Network Resource Management
Hybrid Columnar Compression
Smart Scans of Data Mining model scoring
Exadata Virtual Machines
I/O Analysis in AWR reports
Above slide shows storage CPU growth pattern with various Exadata releases
Below chart shows how raw storage has grown with various generations of Exadata. Usable volume depends on what kind of redundancy you choose during install. Here a key point need to be noted that because of unique Exadata compression feature (EHCC), effective data storage is a lot more than usable volume.
Above slide shows raw disk space growth pattern with various Exadata releases
Data load speed is also growing
Above slide shows data load speed growth pattern with various Exadata releases
Database Server's CPU and Memory
CPU speed and CPU count is directly proportional to database servers processing power where in memory would facilitate any database processing on the database nodes.
While the growing CPU and memory with various Exadata generation is helping database processing and operation, Oracle In-Memory (an Oracle 12c feature) will get most benefit out of it.
Oracle In-Memory option allows storing data in columnar format which give multiple times performance gain for DSS systems where you tend to retrieve fewer columns with multiple rows and this performance enhancement is further multiplied since you retrieve that data from memory instead of slower disks. Traditionally, Oracle stores data in tables in form of rows. This new feature will allow to store data in memory in columnar format.
Above slide shows total database node CPU growth pattern with various Exadata releases
Above slide shows database node memory growth pattern with various Exadata releases
I would like to reiterate that current X5 Exadata series offers extreme system scalability and growth with elastic configurations.
With X5-2 series you can choose between Full/Half/Quarter and Eights Rack.
If you need larger storage capacity, the Oracle Exadata Storage Expansion Rack is available. The Exadata storage expansion rack enables you to grow the Exadata storage.
Capacity-on-Demand allows up to 60% of the cores per server to be turned off during the hardware installation, leaving at least 14 cores enabled. As your workload grows and more cores are needed, Capacity-on-Demand can be used
You can stack multiple racks and they can be connected using the integrated InfiniBand fabric to form even larger configurations. For example, a system composed of four full racks is simply four times as powerful as a single rack system — providing quadruple the I/O throughput, quadruple the storage capacity, and quadruple the processors.
Oracle multitenant, In-Memory, ILM, ADO with Heat Maps add icing on cake when run on Exadata.
Thanks for reading. Happy Exa-dating ! J
***Facts represented in this paper are based on best of my understanding and do not reflect opinion of any company