基于IP SAN的Oracle 11gR2 RAC构建（一）

作者: chimpansee | 来源:发表于2019-06-07 12:41 被阅读0次

11gR2集群又称为GI（Grid Infrastructure），保留了10g的核心功能，在结构上更标准，更精细。所有组件以资源形式存在，统一由代理进程管理。按照功能，可以分为集群初始化组件、集群存储管理组件、应用程序组件。
集群初始化组件主要包括：ohasd守护进程、bootstrap本地节点、ocssd进程。
集群存储管理组件主要负责管理共享存储ASM磁盘组。
应用程序组件负责管理集群中应用资源，包括crsd守护进程。
按照集群功能特点划分，RAC属于一种hot failover 类型的集群。它的每个节点都处于活动状态，都能分担应用负载，某一节点出现问题，负载由其他节点承担，对用户完全透明。
按照数据共享特点划分，RAC属于一种share-everything集群，有自己的共享磁盘架构，有很好的可测量性和扩展性。但是存在数据并发读写的控制问题，需要引入额外的锁机制保证读、写的串行化。

如下是RAC的结构示意图

image.png

二。SAN概述：

FC SAN有着高效、安全的特点，IP SAN有着较低的成本和好的网络适应性的特点。在数据访问要求不是特别高的场合和海量数据存储场合，我们一般采用IP SAN实现数据的远程访问。
Linux I/O早在2011年就成为了Linux Kernel的一部分。它前端支持FCoE, Fibre Channel, IEEE 1394, iSCSI, iSER, SRP and USB等多种互联。这使得LIO成为许多存储供应商的重要选择。我们这里使用LIO支持的iSCSI为数据库提供存储支持。
我们使用LIO的配置工具targetcli来方便的使用iSCSI。

三。集群架构

集群设计采用2台服务器，一台IP SAN服务器。如下图：

arch.jpg

四。安装及调试工具

1.ssh客户端：PuTTY0.70。下载
2.X window 客户端：Xming 6.9及以上版本。下载
3.操作系统：Oracle Linux 7.0版本。下载
4.虚拟化平台：VMware Workstation :12及以上。下载

五。storage服务器安装及配置

1.Oracle Linux 基本安装（省略）
为对外提供存储服务，storage虚拟机上应该添加额外的硬盘。这里我们额外添加了两块20G硬盘。
fdisk -l命令可以看到如下结果：

[root@storage ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        2610    20860402+  8e  Linux LVM
Disk /dev/sdb: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        2610    20964793+  8e  Linux LVM
Disk /dev/sdc: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        2610    20964793+  8e  Linux LVM

2.关闭防火墙
为了简化安装，我们关闭系统防火墙。
如果是使用systemctl的系统了。就用systemctl关闭防火墙,并且disable防火墙:

[root@storage ~]# systemctl stop firewalld
[root@storage ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)

Jun 06 13:59:33 storage systemd[1]: Starting firewalld - dynamic firewall .....
Jun 06 13:59:34 storage systemd[1]: Started firewalld - dynamic firewall d...n.
Jun 06 13:59:47 storage systemd[1]: Stopping firewalld - dynamic firewall .....
Jun 06 13:59:48 storage systemd[1]: Stopped firewalld - dynamic firewall d...n.
Hint: Some lines were ellipsized, use -l to show in full.
[root@storage ~]# systemctl disable firewalld

3.安装ISCSI管理工具
我们就安装和使用LIO targetcli。yum本地源配置如下：

[root@storage ~]# cat /etc/yum.repos.d/local.repo
[local]
name=local
baseurl=file:///mnt/cdrom
gpgcheck=0
enabled=1

安装LIO targetcli。

[root@192 Packages]# yum install  targetcli

检查并启动targetcli，一般随机发布还有一个targetd包，这个是实现基于LIO的远程管理程序包，我们配置完成后，可以试试这个包的使用。

[root@storage ~]# systemctl status target
● target.service - Restore LIO kernel target configuration
   Loaded: loaded (/usr/lib/systemd/system/target.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Thu 2019-06-06 17:10:46 CST; 4s ago
  Process: 22525 ExecStop=/usr/bin/targetctl clear (code=exited, status=0/SUCCESS)
 Main PID: 22464 (code=exited, status=0/SUCCESS)

Jun 06 17:09:00 storage systemd[1]: Starting Restore LIO kernel target configuration...
Jun 06 17:09:00 storage systemd[1]: Started Restore LIO kernel target configuration.
Jun 06 17:10:46 storage systemd[1]: Stopping Restore LIO kernel target configuration...
Jun 06 17:10:46 storage systemd[1]: Stopped Restore LIO kernel target configuration.
[root@storage ~]# systemctl start target
[root@storage ~]# systemctl status target
● target.service - Restore LIO kernel target configuration
   Loaded: loaded (/usr/lib/systemd/system/target.service; enabled; vendor preset: disabled)
   Active: active (exited) since Thu 2019-06-06 17:11:06 CST; 2s ago
  Process: 22525 ExecStop=/usr/bin/targetctl clear (code=exited, status=0/SUCCESS)
  Process: 22550 ExecStart=/usr/bin/targetctl restore (code=exited, status=0/SUCCESS)
 Main PID: 22550 (code=exited, status=0/SUCCESS)

Jun 06 17:11:06 storage systemd[1]: Starting Restore LIO kernel target configuration...
Jun 06 17:11:06 storage systemd[1]: Started Restore LIO kernel target configuration.

4.准备创建pv，vg，lv
由于我们是在建立系统后添加的硬盘，所以需要扫描总线，发现新硬盘。

[root@storage ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0          11:0    1  4.3G  0 rom  /mnt/cdrom
sda           8:0    0   40G  0 disk
├─sda2        8:2    0   39G  0 part
│ ├─ol-swap 252:1    0    2G  0 lvm  [SWAP]
│ └─ol-root 252:0    0   37G  0 lvm  /
└─sda1        8:1    0    1G  0 part /boot
[root@storage ~]# echo "- - -"> /sys/class/scsi_host/host2/scan
[root@storage ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb           8:16   0   40G  0 disk
sr0          11:0    1  4.3G  0 rom  /mnt/cdrom
sda           8:0    0   40G  0 disk
├─sda2        8:2    0   39G  0 part
│ ├─ol-swap 252:1    0    2G  0 lvm  [SWAP]
│ └─ol-root 252:0    0   37G  0 lvm  /
└─sda1        8:1    0    1G  0 part /boot
[root@storage ~]#

建立磁盘分区。

[root@storage ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x5b5b6bcc.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p):
Using default response p
Partition number (1-4, default 1): 1
First sector (2048-83886079, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-83886079, default 83886079):
Using default value 83886079
Partition 1 of type Linux and of size 40 GiB is set

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

创建pv，vg

[root@storage ~]# pvcreate /dev/sdb1
  Physical volume "/dev/sdb1" successfully created.
[root@storage ~]# vgcreate storagevg /dev/sdb1
  Volume group "storagevg" successfully created

创建lv：第一个是lv_data,主要用于数据库文件存储，大小20G，第二个是lv_fra，用于Fast Recovery Area，大小是15G，第三个lv_crs，用于仲裁文件和OCR(Oracle Cluster Register)，大小2G。

[root@storage ~]# lvcreate -L 20G storagevg -n lv_data
  Logical volume "lv_data" created.
[root@storage ~]# lvcreate -L 15G storagevg -n lv_fra
  Logical volume "lv_fra" created.
[root@storage ~]# lvcreate -L 2G storagevg -n lv_crs
  Logical volume "lv_crs" created.
[root@storage ~]# lsblk
NAME                  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb                     8:16   0   40G  0 disk
└─sdb1                  8:17   0   40G  0 part
  ├─storagevg-lv_crs  252:4    0    2G  0 lvm
  ├─storagevg-lv_data 252:2    0   20G  0 lvm
  └─storagevg-lv_fra  252:3    0   15G  0 lvm
sr0                    11:0    1  4.3G  0 rom  /mnt/cdrom
sda                     8:0    0   40G  0 disk
├─sda2                  8:2    0   39G  0 part
│ ├─ol-swap           252:1    0    2G  0 lvm  [SWAP]
│ └─ol-root           252:0    0   37G  0 lvm  /
└─sda1                  8:1    0    1G  0 part /boot
[root@storage ~]#

下面使用targetcli创建block设备：

[root@storage ~]# targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> cd /backstores/block
/backstores/block> create data /dev/storagevg/lv_data
Created block storage object data using /dev/storagevg/lv_data.
/backstores/block> create fra /dev/storagevg/lv_fra
Created block storage object fra using /dev/storagevg/lv_fra.
/backstores/block> create crs /dev/storagevg/lv_crs
Created block storage object crs using /dev/storagevg/lv_crs.
/backstores/block>

创建iqn和lun：

/backstores/block> cd /iscsi
/iscsi> create
Created target iqn.2003-01.org.linux-iscsi.storage.x8664:sn.b0a7e05a08de.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi> cd /iscsi/iqn.2003-01.org.linux-iscsi.storage.x8664:sn.b0a7e05a08de/tpg1/luns
/iscsi/iqn.20...8de/tpg1/luns> create /backstores/block/
/backstores/block/crs   /backstores/block/data  /backstores/block/fra
/iscsi/iqn.20...8de/tpg1/luns> create /backstores/block/data
Created LUN 0.
/iscsi/iqn.20...8de/tpg1/luns> create /backstores/block/fra
Created LUN 1.
/iscsi/iqn.20...8de/tpg1/luns> create /backstores/block/crs
Created LUN 2.
/iscsi/iqn.20...8de/tpg1/luns>

创建acl和portal：

/iscsi/iqn.20...8de/tpg1/luns> cd /iscsi/iqn.2003-01.org.linux-iscsi.storage.x8664:sn.b0a7e05a08de/
/iscsi/iqn.20....b0a7e05a08de> cd tpg1/acls
/iscsi/iqn.20...8de/tpg1/acls> create iqn.2003-01.org.linux-iscsi.storage.x8664:sn.b0a7e05a08de:client
Created Node ACL for iqn.2003-01.org.linux-iscsi.storage.x8664:sn.b0a7e05a08de:client
Created mapped LUN 2.
Created mapped LUN 1.
Created mapped LUN 0.
/iscsi/iqn.20...8de/tpg1/acls> cd ../portals/
/iscsi/iqn.20.../tpg1/portals> ls
o- portals ............................................................................................................ [Portals: 1]
  o- 0.0.0.0:3260 ............................................................................................................. [OK]
/iscsi/iqn.20.../tpg1/portals> delete 0.0.0.0 3260
Deleted network portal 0.0.0.0:3260
/iscsi/iqn.20.../tpg1/portals> create 192.168.237.128
Using default IP port 3260
Created network portal 192.168.237.128:3260.
/iscsi/iqn.20.../tpg1/portals>
/iscsi/iqn.20.../tpg1/portals> cd ..
/iscsi/iqn.20...e05a08de/tpg1> set attribute generate_node_acls=1
Parameter generate_node_acls is now '1'.
/iscsi/iqn.20...e05a08de/tpg1> set attribute authentication=0
Parameter authentication is now '0'.
/iscsi/iqn.20...e05a08de/tpg1> set attribute demo_mode_write_protect=0
Parameter demo_mode_write_protect is now '0'.

启动时启动ISCSID服务。

[root@192 ~]# systemctl enable iscsid
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.

支持IPSAN配置完毕。下一节我们开始集群节点的配置。

基于IP SAN的Oracle 11gR2 RAC构建（一）

目录

一。11g RAC概述：

二。SAN概述：

三。集群架构

四。安装及调试工具

五。storage服务器安装及配置

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读