back

next

 

 

 

 

 

How to enable Exadata writeback Flashcache on cell storage ? (WriteThrough to WriteBack)

 

 

There are two key features of the Exadata Storage Server Software that leverage the Exadata Flash hardware and make the Exadata Database Machine such a fast system on which to deploy the Oracle Database. First is Exadata Smart Flash Cache which provides the capability to stage active database objects in flash. Second is the Exadata Smart Flash Logging which speeds the critical function of database logging. Lastly, the deployment of the Oracle Database requires mission critical resilience and the Exadata Storage Server Software in conjunction with the Oracle Database provides that.

 

 

              

 

High Availability

There are two key features of the Exadata Storage Server Software that leverage the Exadata Flash hardware and make the Exadata Database Machine such a fast system on which to deploy the Oracle Database. First is Exadata Smart Flash Cache which provides the capability to stage active database objects in flash. Second is the Exadata Smart Flash Logging which speeds the critical function of database logging. Lastly, the deployment of the Oracle Database requires mission critical resilience and the Exadata Storage Server Software in conjunction with the Oracle Database provides that. In addition to improving read I/Os, Oracle Exadata Write back flash cache also provides the ability to cache write I/Os directly to PCI flash.

 

 

 

              

 

 

System requirements

Exadata storage software version 11.2.3.2.1 is the minimum version required to use write back flash cache. Grid infrastructure and database homes must run 11.2.0.3.9 or later to use with Write-back Smart Flash Cache. Database homes running 11.2.0.2 must install Patch 17342825.

 

Write-Back Flash Cache Benefits:

 

  • It improves the write intensive operations because writing to flash cache is much faster than writing to Hard disks.
  • Exadata X4-2 Database Machines write performance can be improved up to 20X IOPS than Hard disk.
  • Exadata X4-2 / V2 Database Machines write performance can be improved up to 10X IOPS than Hard disk.
  • Write-Back Flash Cache transparently accelerates reads and writes for all workloads for OLTP (faster random reads and writes) and DW (faster sequential smart scans).
  • Write-Back Flash Cache reduce latency of redo log writes when it shares disks with data.
  • Data recoverable from Flash cache on cellsrv restart
  • If you find significant waits for "free buffer waits" or high IO times to check for write bottleneck in AWR reports, then you should consider using the write back flash cache feature.

 

 Enable Write-Back Flash Cache:  Methods are available:

 

  • Rolling Method / Assuming that RDBMS & ASM instances are UP and enabling Write-Back Flash Cache in One Cell Server at a time
  • Non-Rolling Method / Assuming that RDBMS & ASM instances are DOWN while enabling Write-Back Flash Cache

 

 

Non-Rolling Method

 

First check flashcache mode with below command on all cell servers with the use of dcli utility on DB node. It can be check on cell each server also with cellcli utility.

 

[root@bakuexa1celadm01 ~]dcli -l root -g cellgroup "cellcli -e list cell detail" | grep "flashCacheMode"

bakuexa1celadm01: WriteThrough

bakuexa1celadm02: WriteThrough

bakuexa1celadm03: WriteThrough

 

Step 1. Drop the flash cache on that cell

 

CellCLI> drop flashcache

Flash cache bakuexa1celadm01_FLASHCACHE successfully dropped

 

Step 2. Shut down cellsrv service

 

The preceding command checks if any disks are offline, in predictive failure status or need to be copied to its mirror. If Oracle ASM redundancy is intact, then the command takes the grid disks offline in Oracle ASM, and then stops the cell services. If the following error is displayed, then it may not be safe to stop the cell services because a disk group may be forced to dismount due to redundancy.

 

CellCLI> alter cell shutdown services cellsrv

Stopping CELLSRV services...

The SHUTDOWN of CELLSRV services was successful.

 

Step 3. Set the cell flashcache mode to writeback

 

CellCLI> alter cell flashCacheMode = WriteBack

Cell bakuexa1celadm01 successfully altered

 

Step 4. Restart the cellsrv service

 

CellCLI> alter cell startup services cellsrv

Starting CELLSRV services...

The STARTUP of CELLSRV services was successful.

 

Step 5. Create the flash cache all

 

 If the flash disk is used for flash cache, then the effective cache size increases. If the flash disk is used for grid disks, then the grid disks are re-created on the new flash disk. If those gird disks were part of an Oracle ASM disk group, then they are added back to the disk group, and the data is rebalanced on them based on the disk group redundancy and ASM_POWER_LIMIT parameter.

 

CellCLI> alter cell startup services cellsrv

Starting CELLSRV services...

The STARTUP of CELLSRV services was successful.

 

 advertisement

 

Rolling Method:

 

Step 1. Drop the flash cache on that cell

 

CellCLI> drop flashcache

Flash cache bakuexa1celadm01_FLASHCACHE successfully dropped

 

Step 2. Check the status of ASM if the grid disks go OFFLINE. The following command should return 'Yes' for the grid disks being listed:

 

CellCLI> cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

         DATAC1_CD_00_bakuexa1celadm01   OFFLINE  Yes

         DATAC1_CD_01_bakuexa1celadm01   OFFLINE  Yes

         DATAC1_CD_02_bakuexa1celadm01   OFFLINE  Yes

         DATAC1_CD_03_bakuexa1celadm01   OFFLINE  Yes

         DATAC1_CD_04_bakuexa1celadm01   OFFLINE  Yes

         DATAC1_CD_05_bakuexa1celadm01   OFFLINE  Yes

         DBFS_DG_CD_02_bakuexa1celadm01  OFFLINE  Yes

         DBFS_DG_CD_03_bakuexa1celadm01  OFFLINE  Yes

         DBFS_DG_CD_04_bakuexa1celadm01  OFFLINE  Yes

         DBFS_DG_CD_05_bakuexa1celadm01  OFFLINE  Yes

         RECOC1_CD_00_bakuexa1celadm01   OFFLINE  Yes

         RECOC1_CD_01_bakuexa1celadm01   OFFLINE  Yes

         RECOC1_CD_02_bakuexa1celadm01   OFFLINE  Yes

         RECOC1_CD_03_bakuexa1celadm01   OFFLINE  Yes

         RECOC1_CD_04_bakuexa1celadm01   OFFLINE  Yes

         RECOC1_CD_05_bakuexa1celadm01   OFFLINE  Yes

 

Step 3. Inactivate the griddisk on the cell

 

CellCLI> cellcli –e alter griddisk all inactive

 

Step 4. Shut down cellsrv service

 

CellCLI> alter cell shutdown services cellsrv

Stopping CELLSRV services...

The SHUTDOWN of CELLSRV services was successful.

 

Step 5. Set the cell flashcache mode to writeback

 

CellCLI> alter cell flashCacheMode = WriteBack

Cell bakuexa1celadm01 successfully altered

 

Step 6. Restart the cellsrv service

 

CellCLI> alter cell startup services cellsrv

Starting CELLSRV services...

The STARTUP of CELLSRV services was successful.

 

Step 7. Reactivate the griddisks on the cell

 

CellCLI> cellcli –e alter griddisk all active

CellCLI> ALTER GRIDDISK ALL ACTIVE

GridDisk DATAC1_CD_00_bakuexa1celadm03 successfully altered

GridDisk DATAC1_CD_01_bakuexa1celadm03 successfully altered

GridDisk DATAC1_CD_02_bakuexa1celadm03 successfully altered

GridDisk DATAC1_CD_03_bakuexa1celadm03 successfully altered

GridDisk DATAC1_CD_04_bakuexa1celadm03 successfully altered

GridDisk DATAC1_CD_05_bakuexa1celadm03 successfully altered

GridDisk DBFS_DG_CD_02_bakuexa1celadm03 successfully altered

GridDisk DBFS_DG_CD_03_bakuexa1celadm03 successfully altered

GridDisk DBFS_DG_CD_04_bakuexa1celadm03 successfully altered

GridDisk DBFS_DG_CD_05_bakuexa1celadm03 successfully altered

GridDisk RECOC1_CD_00_bakuexa1celadm03 successfully altered

GridDisk RECOC1_CD_01_bakuexa1celadm03 successfully altered

GridDisk RECOC1_CD_02_bakuexa1celadm03 successfully altered

GridDisk RECOC1_CD_03_bakuexa1celadm03 successfully altered

GridDisk RECOC1_CD_04_bakuexa1celadm03 successfully altered

GridDisk RECOC1_CD_05_bakuexa1celadm03 successfully altered

 

Step 8. Verify all grid disks have been successfully put online using the following command:

 

CellCLI> list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

         DATAC1_CD_00_bakuexa1celadm02   ONLINE         Yes

         DATAC1_CD_01_bakuexa1celadm02   ONLINE         Yes

         DATAC1_CD_02_bakuexa1celadm02   ONLINE         Yes

         DATAC1_CD_03_bakuexa1celadm02   ONLINE         Yes

         DATAC1_CD_04_bakuexa1celadm02   ONLINE         Yes

         DATAC1_CD_05_bakuexa1celadm02   ONLINE         Yes

         DBFS_DG_CD_02_bakuexa1celadm02  ONLINE         Yes

         DBFS_DG_CD_03_bakuexa1celadm02  ONLINE         Yes

         DBFS_DG_CD_04_bakuexa1celadm02  ONLINE         Yes

         DBFS_DG_CD_05_bakuexa1celadm02  ONLINE         Yes

         RECOC1_CD_00_bakuexa1celadm02   ONLINE         Yes

         RECOC1_CD_01_bakuexa1celadm02   ONLINE         Yes

         RECOC1_CD_02_bakuexa1celadm02   ONLINE         Yes

         RECOC1_CD_03_bakuexa1celadm02   ONLINE         Yes

         RECOC1_CD_04_bakuexa1celadm02   ONLINE         Yes

         RECOC1_CD_05_bakuexa1celadm02   ONLINE         Yes

 

Step 9. Recreate the flash cache

 

CellCLI> create flashcache all

Flash cache bakuexa1celadm01_FLASHCACHE successfully created

 

Step 10. Check the status of the cell to confirm that it's now in WriteBack mode:

 

CellCLI> cellcli -e list cell detail | grep flashCacheMode

flashCacheMode:         WriteBack

 

Step 11.  Repeat these same steps again on the next cell to the FINAL cell. However, before taking another storage server offline, execute the following making sure 'asmdeactivationoutcome' displays YES:

 

CellCLI> cellcli -e list griddisk attributes name,asmmodestatus,asmdeactivationoutcome

        DATAC1_CD_00_bakuexa1celadm01   ONLINE  Yes

         DATAC1_CD_01_bakuexa1celadm01   ONLINE  Yes

         DATAC1_CD_02_bakuexa1celadm01   ONLINE  Yes

         DATAC1_CD_03_bakuexa1celadm01   ONLINE  Yes

         DATAC1_CD_04_bakuexa1celadm01   ONLINE  Yes

         DATAC1_CD_05_bakuexa1celadm01   ONLINE  Yes

         DBFS_DG_CD_02_bakuexa1celadm01  ONLINE  Yes

         DBFS_DG_CD_03_bakuexa1celadm01  ONLINE  Yes

         DBFS_DG_CD_04_bakuexa1celadm01  ONLINE  Yes

         DBFS_DG_CD_05_bakuexa1celadm01  ONLINE  Yes

         RECOC1_CD_00_bakuexa1celadm01   ONLINE  Yes

         RECOC1_CD_01_bakuexa1celadm01   ONLINE  Yes

         RECOC1_CD_02_bakuexa1celadm01   ONLINE  Yes

         RECOC1_CD_03_bakuexa1celadm01   ONLINE  Yes

         RECOC1_CD_04_bakuexa1celadm01   ONLINE  Yes

         RECOC1_CD_05_bakuexa1celadm01   ONLINE  Yes

 

After changing the flashcache modes on all cells, check if flashcache modes are changed to write-back for all cells.

 

CellCLI> dcli -g ~/cell_group -l root cellcli -e "list cell attributes flashcachemode"

bakuexa1celadm01: WriteBack

bakuexa1celadm02: WriteBack

bakuexa1celadm03: WriteBack

 

 

Summary

 

Use the Write-Back Flash Cache feature to leverage the Exadata Flash hardware and make Exadata Database Machine a faster system for Oracle Database Deployments.  Flash Storage inside the Oracle Exadata Database Machine is used completely as Flash Cache by default, effectively working as an extension of the Database Buffer Cache  and delivering faster Access together with a very high IO per Second rate which is especially important for OLTP. Additionally, we may take a part of the Flash Storage to build ASM diskgroups upon it. Files placed on these diskgroups will reside permanently on Flash Storage – no Caching needed.

OTECH MAGAZINE #8  summer 2015        copyright otech magazine 2015

www.otechmag.com