Thanks to my coworkers David Karlsen and Lars Johan Ulveseth for introducing me to the world of Vagrant & SALT, and letting me use some of their work on this.
Earlier post:
Sometimes it’s alright to be LAZY – Part 1
In part 1 we looked at the tools Oracle VirtualBox and Vagrant, and how we could use these tools to easily manage our test databases. In this blog post we’ll take a look at the provisioning tool SALT, and how to use this to easily build our Oracle Vagrant boxes.
In this blog I’ll not use the remote functionality of the product, and only run it locally. The article only show a very simple and narrow use of the tool, and should not be considered a introduction to SALT. My goal is only to show how SALT can be used to generate a pre-build Oracle database vagrant box. For an introduction to SALT please read the SaltStack documentation.
Salt documentation makes this 30-second summary of their tool:
Salt is:
- a configuration management system, capable of maintaining remote nodes in defined states (for example, ensuring that specific packages are installed and specific services are running)
- a distributed remote execution system used to execute commands and query data on remote nodes, either individually or by arbitrary selection criteria
It was developed in order to bring the best solutions found in the world of remote execution together and make them better, faster, and more malleable. Salt accomplishes this through its ability to handle large loads of information, and not just dozens but hundreds and even thousands of individual servers quickly through a simple and manageable interface.
As I stated earlier in this post, we’ll only look at a very simple and narrow use of the SaltStack tools. If you want to download, use and/or contribute to the framework presented in this blog, you’ll find the scripts at my github repository.
First I’ll show the tree structure of the SALT framework:
$ git clone https://github.com/lasjen/OracleVagrantFramework.git $ cd OracleVagrantFramework $ tree . ├── Makefile ├── minion ├── srv │ ├── pillar │ │ ├── oracle12c │ │ │ └── init.sls │ │ └── top.sls │ └── salt │ ├── oracle12c │ │ ├── 12c_cfgrsp.properties │ │ ├── 12c_oracle_EE.rsp │ │ ├── 12c_pdb_autostart.sh │ │ ├── alteruser.sql │ │ ├── create_dev_users.sql │ │ ├── dev_users_env.sql │ │ ├── drop_dev_users.sql │ │ ├── etc │ │ │ ├── fstab │ │ │ ├── init.d │ │ │ │ └── dbora │ │ │ └── security │ │ │ └── limits.d │ │ │ └── 90-nproc.conf │ │ ├── home │ │ │ └── oracle │ │ │ └── scripts │ │ │ ├── shutdown.sh │ │ │ ├── startup.sh │ │ │ └── upgrade_pdb.sh │ │ ├── init.sls │ │ ├── map.jinja │ │ ├── noexpirepw.sql │ │ ├── ocm.rsp │ │ ├── patch.sls │ │ ├── shrink_tablespaces_part1.sql │ │ ├── shrink_tablespaces_part2.sql │ │ ├── u01 │ │ │ └── app │ │ │ └── oracle │ │ │ └── product │ │ │ └── 12.1.0 │ │ │ └── dbhome_1 │ │ │ └── network │ │ │ └── admin │ │ │ ├── listener.ora │ │ │ └── tnsnames.ora │ │ └── vktm_highcpu_vbox_hack.sql │ └── top.sls ├── uploadboxtoshare.sh └── Vagrantfile 20 directories, 30 files
Here I make use of the “make” utility, and the configuration in the “Makefile”:
export WORKSPACE=/backup VAGRANT_HOME=$(WORKSPACE)/.vagrant.d VBOX_USER_HOME=$(WORKSPACE)/.vbox EXPORT_BOX_FILE=vagrant-oracle12c-ee-`git rev-list --max-count=1 HEAD`.box export http_proxy= export https_proxy=$(http_proxy) clean: vagrant destroy -f rm -f *box build: setup vagrant up export: build vagrant halt vagrant package --output $(EXPORT_BOX_FILE) vagrant destroy -f upload: export ./uploadboxtoshare.sh $(EXPORT_BOX_FILE) all: upload setup: clean set vagrant --version VBoxManage --version VBoxManage setproperty machinefolder $(VBOX_USER_HOME)/machinefolder VBoxManage setproperty hwvirtexclusive off vagrant plugin install vagrant-proxyconf vagrant plugin install vagrant-vbguest vagrant plugin install vagrant-reload
To build the box I now need to run the following command:
$ make export
This will “clean”, “setup”, “build”, and “export”. In plain English this is:
“Vagrant will destroy the existing old virtual machine (if any), and remove the box. Then we set some configuration with VBoxManage, and make sure the Vagrant plugins are in place. Vagrant will then startup the new machine reading the Vagrantfile (This is where all the action is done by Salt building our new Oracle Vagrant box). After the machine is build the machine is halted, and a vagrant box is packaged.”
Vagrantfile
The step that really does all the work is the “vagrant up” command under the build step. As we saw in part 1, during the vagrant up command, Vagrant acts on the content in the Vagrantfile:
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vbguest.auto_update = true
config.vm.box = "boxcutter/ol67"
config.vm.box_version = "= 3.0.3"
config.vm.hostname = "oracle"
if ENV['VAGRANT_ORACLE_PORT']
oraclePort = ENV['VAGRANT_ORACLE_PORT'].to_i
else
oraclePort = 1521
end
# Forward Oracle port
config.vm.network :forwarded_port, guest: 1521, host: oraclePort
config.vm.provider :virtualbox do |vb|
vb.cpus = 2
# Use VBoxManage to customize the VM
vb.customize ["modifyvm", :id,
"--name", "oracle",
"--memory", "2048",
"--cpus", "1",
# Enable DNS behind NAT
"--natdnshostresolver1", "on"]
end
config.vm.synced_folder "srv", "/srv/"
config.vm.synced_folder ".", "/vagrant"
#enable repos, update packages, install salt-minion:
config.vm.provision :shell, :inline => "yum install -y yum-utils && yum-config-manager --enable epel\* && yum -y update && rpm -Uvh http://ftp.linux.ncsu.edu/pub/epel/6Server/x86_64/epel-release-6-8.noarch.rpm"
#needed for virtualbox guest kernel stuff:
config.vm.provision :shell, :inline => "yum install -y kernel-uek-devel kernel-devel"
config.vm.provision :salt do |salt|
#salt.colorize = true
#salt.log_level = "all"
salt.verbose = true
#salt.no_minion = true
#https://github.com/mitchellh/vagrant/issues/5973:
#salt.minion_config = "vagrant/minion"
# pillar data is in vagrant/pillar/top.sls|wxs.sls
#https://github.com/mitchellh/vagrant/issues/5973:
#salt.run_highstate = true
end
#https://github.com/mitchellh/vagrant/issues/5973
config.vm.provision :shell, :inline => "sudo cp /vagrant/minion /etc/salt/minion && sudo service salt-minion restart && salt-call state.highstate"
config.vm.provision :reload
config.vm.provision :shell, :inline => "sudo salt-call state.sls oracle12c.patch"
#cleanup:
config.vm.provision :shell, :inline => "yum clean all"
config.vm.provision :shell, :inline => "salt-call saltutil.clear_cache"
config.vm.provision :shell, :inline => "dd if=/dev/zero of=/zero.fill bs=1M;sync;sleep 1;sync;rm -f /zero.fill"
end
Some of this you’ll recognize from the part 1. I’ll not cover this again. The installation is based on a existing box found on atlas.hashicorp.com:
config.vm.box = "boxcutter/ol67" config.vm.box_version = "= 3.0.3" config.vm.hostname = "oracle"
By setting the BOX_VERSION we know that we don’t get any surprises caused by changes done on the box. We also define a very simplified name on the box.
Then it´s time to start the installation of the needed packages on our virtual machine (Linux repos etc), and most importantly, installing, configuring and starting Salt.
#enable repos, update packages, install salt-minion:
config.vm.provision :shell, :inline => "yum install -y yum-utils && yum-config-manager --enable epel\* && yum -y update && rpm -Uvh http://ftp.linux.ncsu.edu/pub/epel/6Server/x86_64/epel-release-6-8.noarch.rpm"
#needed for virtualbox guest kernel stuff:
config.vm.provision :shell, :inline => "yum install -y kernel-uek-devel kernel-devel"
config.vm.provision :salt do |salt|
#salt.colorize = true
#salt.log_level = "all"
salt.verbose = true
#salt.no_minion = true
#https://github.com/mitchellh/vagrant/issues/5973:
#salt.minion_config = "vagrant/minion"
# pillar data is in vagrant/pillar/top.sls|wxs.sls
#https://github.com/mitchellh/vagrant/issues/5973:
#salt.run_highstate = true
end
#https://github.com/mitchellh/vagrant/issues/5973
config.vm.provision :shell, :inline => "sudo cp /vagrant/minion /etc/salt/minion && sudo service salt-minion restart && salt-call state.highstate"
The Salt configuration is found in the minion file (discussed below in section “Salt setup”). The command “salt-call state.highstate” triggers Salt to start the provisioning. This will be described in further details in the section “The Salt provisioning” (see below).
After Salt have completed this provisioning (highstate), the Oracle software will be installed and an Oracle database will be running on our virtual machine. The next step is to apply needed patches. Then we use Salt again, to start yet another provisioning (using the same techniques used in the first provisioning). Before applying the path we need to do a restart of our VM. To do this we use the reload plugin (vagrant). The details will be explained in the section “Extending the provisioning to apply Oracle patches” (at the bottom).
#then reload so we can... config.vm.provision :reload config.vm.provision :shell, :inline => "sudo salt-call state.sls oracle12c.patch"
The final part of the Vagrantfile does some cleaning up. We really want the size of our base box to be as smal as possible. So we try to remove as much space we can from the VM:
#cleanup: config.vm.provision :shell, :inline => "yum clean all" config.vm.provision :shell, :inline => "salt-call saltutil.clear_cache" config.vm.provision :shell, :inline => "dd if=/dev/zero of=/zero.fill bs=1M;sync;sleep 1;sync;rm -f /zero.fill"
Salt setup
As we saw earlier, the Vagrantfile triggers the enabling of Linux repos, update of packages, installation of salt-minion, startup of the salt-minion daemon on our guest machine, and finally Salt calling the “state.highstate” (which will start the provisioning). Vagrant also copied the minion file to /etc/salt directory. This is where the configuration of SALT takes place. The minion file is read at the salt-minion daemon startup, and looks like this:
state_top: top.sls file_client: local file_roots: base: - /srv/salt pillar_roots: base: - /srv/pillar
“state_top” defines which file SALT should start with when calling the highstate. “file_client: local” defines that we are using standalone minions, and tells SALT to run local only (no masters involved). Then we set the file and pillar paths (root) for the “base” environments (just a name for our environment). In our setup the pillar part only include an “empty” configuration. The rest of the blog will focus on the “/srv/salt”-part and -files. As the above configuration defines, SALT will now read the /srv/salt/top.sls file:
base:
'*':
-oracle12c
This tells SALT that for all (‘*’) minions run the provisioning defines by “oracle12c/init.sls” for the “base” environment. The “init.sls” is the default file SALT will look for.
The Salt provisioning
The “oracle12c/init.sls” does the real provisioning. This is where all the power and action of SALT really is defined.
So … let’s have a look at the “init.sls” file:
{% from "oracle12c/map.jinja" import extractdir, ora_app, ora_prod with context %}
{% set ora_d = extractdir ~ '/ora' %}
unzip:
pkg.installed
/etc/security/limits.d/90-nproc.conf:
file.managed:
- source: salt://oracle12c/etc/security/limits.d/90-nproc.conf
/etc/fstab:
file.managed:
- source: salt://oracle12c/etc/fstab
#yet again salt fails on file.managed... do a wget instead:
wget -O /tmp/p17694377_121020_Linux-x86-64_1of8.zip http:///p17694377_121020_Linux-x86-64_1of8.zip:
cmd.run
wget -O /tmp/p17694377_121020_Linux-x86-64_2of8.zip http:///p17694377_121020_Linux-x86-64_2of8.zip:
cmd.run
mkdir -p {{ora_d}} && unzip /tmp/p17694377_121020_Linux-x86-64_1of8.zip -d {{ora_d}} && unzip /tmp/p17694377_121020_Linux-x86-64_2of8.zip -d {{ora_d}} && rm /tmp/p17694377_121020_Linux-x86-64_1of8.zip /tmp/p17694377_121020_Linux-x86-64_2of8.zip:
cmd.run
#https://github.com/saltstack/salt/issues/23822:
chmod -R a+x {{extractdir}}:
cmd.run
oracle-rdbms-server-12cR1-preinstall:
pkg.installed
{% for file in [ '12c_oracle_EE.rsp','12c_cfgrsp.properties' ] %}
{{extractdir}}/{{file}}:
file.managed:
- source: salt://oracle12c/{{file}}
{% endfor %}
{{ora_app}}:
file.directory:
- user: oracle
- group: oinstall
- mode: 775
- makedirs: True
#need to -ignorePrereq since numprocesses are not yet read by this shell:
{{ora_d}}/database/runInstaller -silent -ignorePrereq -waitforcompletion -ignoreSysPrereqs -responseFile {{extractdir}}/12c_oracle_EE.rsp:
cmd.run:
- user: oracle
/u01/app/oraInventory/orainstRoot.sh:
cmd.run
{{ora_prod}}/dbhome_1/root.sh:
cmd.run
{{ora_prod}}/dbhome_1/cfgtoollogs/configToolAllCommands RESPONSE_FILE={{extractdir}}/12c_cfgrsp.properties:
cmd.run:
- user: oracle
/tmp/12c_pdb_autostart.sh:
file.managed:
- source: salt://oracle12c/12c_pdb_autostart.sh
- mode: 0755
- user: oracle
- group: oinstall
cmd.run:
- user: oracle
/etc/init.d/dbora:
file.managed:
- source: salt://oracle12c/etc/init.d/dbora
- user: root
- group: root
- mode: 755
chkconfig --add dbora:
cmd.run
/home/oracle/.profile:
file.managed:
- source: salt://oracle12c/home/oracle/.profile
- user: oracle
- group: oinstall
#test, create .bash_profile, since .profile will not be read if .bash_profile present and using bash
/home/oracle/.bash_profile:
file.managed:
- source: salt://oracle12c/home/oracle/.profile
- user: oracle
- group: oinstall
/home/oracle/scripts:
file.recurse:
- source: salt://oracle12c/home/oracle/scripts
- user: oracle
- file_mode: 0755
#start database so we can run sqlplus on it afterwords:
/etc/init.d/dbora start:
cmd.run
#I don't think vktm_highcpu_vbox_hack is needed - let's try withoug
{% for script in [ 'noexpirepw', 'create_dev_users', 'alteruser' ] %}
/tmp/{{script}}.sql:
file.managed:
- source: salt://oracle12c/{{script}}.sql
{{ora_prod}}/dbhome_1/bin/sqlplus "/ as sysdba" @/tmp/{{script}}.sql > /tmp/{{script}}.log:
cmd.run:
- user: oracle
{% endfor %}
/tmp/shrink_tablespaces_part1.sql:
file.managed:
- source: salt://oracle12c/shrink_tablespaces_part1.sql
/tmp/shrink_tablespaces_part2.sql:
file.managed:
- source: salt://oracle12c/shrink_tablespaces_part2.sql
# Network Config
{{ora_prod}}/dbhome_1/network/admin/listener.ora
file.managed:
- source: salt://oracle12c/u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
{{ora_prod}}/dbhome_1/network/admin/tnsnames.ora
file.managed:
- source: salt://oracle12c/u01/app/oracle/product/12.1.0/dbhome_1/network/admin/tnsnames.ora
{{ora_prod}}/dbhome_1/bin/sqlplus / as sysdba @/tmp/shrink_tablespaces_part1.sql > /tmp/shrink_tablespaces.log:
cmd.run:
- user: oracle
restartDB_ts:
cmd.run:
- user: oracle
- name: /home/oracle/scripts/shutdown.sh && /home/oracle/scripts/startup.sh
{{ora_prod}}/dbhome_1/bin/sqlplus / as sysdba @/tmp/shrink_tablespaces_part2.sql >> /tmp/shrink_tablespaces.log:
cmd.run:
- user: oracle
#cleanup
{{extractdir}}:
file.absent
Let’s go through the provisioning step-by-step:
{% from "oracle12c/map.jinja" import extractdir, ora_app, ora_prod with context %}
{% set ora_d = extractdir ~ '/ora' %}
unzip:
pkg.installed
First (the first two lines) – SALT sets some path parameters (extractdir, ora_app, ora_prod and ora_d). These will be used further down in the provisioning.
Note! The map.jinja is pretty easy to read. The piller.get method looks for a pillar (read: pillar variable) with the name (in first argument). If not found it will return the second argument.
{% set extractdir = salt[‘pillar.get’](‘ora_extractdir’, ‘/var/install_files’) %}
If you want to override this extractdir, just define a pillar in in your file /srv/pillar/oracle12c/init.sls:
base:
´*´:
ora_extractdir: <your_path>
In the 3rd and 4th line SALT uses the pkg.installed pseudocode to make sure that the “unzip” package is installed.
Then we copy some files into the environment by using the file.managed pseudocode.
/etc/security/limits.d/90-nproc.conf:
file.managed:
- source: salt://oracle12c/etc/security/limits.d/90-nproc.conf
/etc/fstab:
file.managed:
- source: salt://oracle12c/etc/fstab
We also tried to use the file.managed or archive pseudocode to download the Oracle software installation files from our local webserver, but the download process terminates randomly. Instead we used the cmd.run pseudocode together with wget and unzip. This is not environment independent, and is not really a good and salty way to do things. But we are only running this on one environment so it doesn’t really matter.
#yet again salt fails on file.managed... do a wget instead:
wget -O /tmp/p17694377_121020_Linux-x86-64_1of8.zip http:///p17694377_121020_Linux-x86-64_1of8.zip:
cmd.run
wget -O /tmp/p17694377_121020_Linux-x86-64_2of8.zip http:///p17694377_121020_Linux-x86-64_2of8.zip:
cmd.run
mkdir -p {{ora_d}} && unzip /tmp/p17694377_121020_Linux-x86-64_1of8.zip -d {{ora_d}} && unzip /tmp/p17694377_121020_Linux-x86-64_2of8.zip -d {{ora_d}} && rm /tmp/p17694377_121020_Linux-x86-64_1of8.zip /tmp/p17694377_121020_Linux-x86-64_2of8.zip:
cmd.run
#https://github.com/saltstack/salt/issues/23822:
chmod -R a+x {{extractdir}}:
cmd.run
We also tried the archive pseudocode, but didn’t make it work:
#unzip.linuxamd64_12c_database_1of2.zip:
# archive:
# - extracted
# - name: {{ora_d}}
# - source: http:///p17694377_121020_Linux-x86-64_1of8.zip
# - source_hash: md5=017d624abe5165406d3d7c7d2d3a834c
# - archive_format: zip
# - require:
# - pkg: unzip
Then it’s time for preparing for the Oracle software installation. We run the pre-install installation, copy the needed responsefiles into our environment, and set needed directory privileges:
oracle-rdbms-server-12cR1-preinstall:
pkg.installed
{% for file in [ '12c_oracle_EE.rsp','12c_cfgrsp.properties' ] %}
{{extractdir}}/{{file}}:
file.managed:
- source: salt://oracle12c/{{file}}
{% endfor %}
{{ora_app}}:
file.directory:
- user: oracle
- group: oinstall
- mode: 775
- makedirs: True
Then we run the Oracle software installation and database creation (defined in the response file):
#need to -ignorePrereq since numprocesses are not yet read by this shell:
{{ora_d}}/database/runInstaller -silent -ignorePrereq -waitforcompletion -ignoreSysPrereqs -responseFile {{extractdir}}/12c_oracle_EE.rsp:
cmd.run:
- user: oracle
/u01/app/oraInventory/orainstRoot.sh:
cmd.run
{{ora_prod}}/dbhome_1/root.sh:
cmd.run
{{ora_prod}}/dbhome_1/cfgtoollogs/configToolAllCommands RESPONSE_FILE={{extractdir}}/12c_cfgrsp.properties:
cmd.run:
- user: oracle
The last call to configToolAllCommands resets the passwords found in the 12c_cfgrsp.properties file.
Next we prepare the environment to startup the database on OS restart, by copying some files,directories and running “chkconfig”:
/tmp/12c_pdb_autostart.sh:
file.managed:
- source: salt://oracle12c/12c_pdb_autostart.sh
- mode: 0755
- user: oracle
- group: oinstall
cmd.run:
- user: oracle
/etc/init.d/dbora:
file.managed:
- source: salt://oracle12c/etc/init.d/dbora
- user: root
- group: root
- mode: 755
chkconfig --add dbora:
cmd.run
/home/oracle/.profile:
file.managed:
- source: salt://oracle12c/home/oracle/.profile
- user: oracle
- group: oinstall
#test, create .bash_profile, since .profile will not be read if .bash_profile present and using bash
/home/oracle/.bash_profile:
file.managed:
- source: salt://oracle12c/home/oracle/.profile
- user: oracle
- group: oinstall
/home/oracle/scripts:
file.recurse:
- source: salt://oracle12c/home/oracle/scripts
- user: oracle
- file_mode: 0755
Then we start the database and run some scripts to set noexpire passwords, create application users, and reduce the TEMP and UNDO tablespaces:
#start database so we can run sqlplus on it afterwords:
/etc/init.d/dbora start:
cmd.run
#I don't think vktm_highcpu_vbox_hack is needed - let's try withoug
{% for script in [ 'noexpirepw', 'create_dev_users', 'alteruser' ] %}
/tmp/{{script}}.sql:
file.managed:
- source: salt://oracle12c/{{script}}.sql
{{ora_prod}}/dbhome_1/bin/sqlplus "/ as sysdba" @/tmp/{{script}}.sql > /tmp/{{script}}.log:
cmd.run:
- user: oracle
{% endfor %}
/tmp/shrink_tablespaces_part1.sql:
file.managed:
- source: salt://oracle12c/shrink_tablespaces_part1.sql
/tmp/shrink_tablespaces_part2.sql:
file.managed:
- source: salt://oracle12c/shrink_tablespaces_part2.sql
{{ora_prod}}/dbhome_1/bin/sqlplus / as sysdba @/tmp/shrink_tablespaces_part1.sql > /tmp/shrink_tablespaces.log:
cmd.run:
- user: oracle
restartDB_ts:
cmd.run:
- user: oracle
- name: /home/oracle/scripts/shutdown.sh && /home/oracle/scripts/startup.sh
{{ora_prod}}/dbhome_1/bin/sqlplus / as sysdba @/tmp/shrink_tablespaces_part2.sql >> /tmp/shrink_tablespaces.log:
cmd.run:
- user: oracle
At last we copy the listener.ora and tnsnames.ora for the network configuring:
# Network Config
{{ora_prod}}/dbhome_1/network/admin/listener.ora
file.managed:
- source: salt://oracle12c/u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora
{{ora_prod}}/dbhome_1/network/admin/tnsnames.ora
file.managed:
- source: salt://oracle12c/u01/app/oracle/product/12.1.0/dbhome_1/network/admin/tnsnames.ora
Then the only thing left is to clean up the environment. For this we use the file.absent pseudocode to delete the files under the EXTRACTDIR directory (the Oracle source files):
#cleanup
{{extractdir}}:
file.absent
Pushing the button:
So the only think left know is actually to push the button:
$ make build
Go and get a cup of coffee or a nice Ambere Ale, and wait for SALT and vagrant to complete it’s job. In about 20-30 minuttes (dependant on the speed of your host machine) your new vagrant box will be ready.
Extending provisioning to install Oracle patch
We also tested out applying a patch with SALT, by adding another Salt provisioning in our original Vagrantfile.
To do this we first had to restart (reload) the VM, and then call our patch SALT state script (…/srv/salt/oracle12c/patch.sls) from the Vagrantfile:
config.vm.provision :reload config.vm.provision :shell, :inline => "sudo salt-call state.sls oracle12c.patch"
For further understanding have a look at the file “patch.sls”. This provisioning file uses the same type of pseudocodes as described in the “oracle12c/init.sls” file.
Post a Comment