Ansible Certification : 2. Deploying Ansible

2. Deploying Ansible. 

• Installing Ansible.
• Managing Ansible Configuration Files.
• Running Ad Hoc Commands.
• Managing Dynamic Inventories.
• Summary
• Lab:
• Exercise

=========================================================================

Installing Ansible 

Ansible installation is relatively simple. All it needs is that you have python 2 or 2.6 version present on the server where its about to be installed. On the managed hosts 2.4 and above will be good. 

Just in case if the version of python installed is earlier than 2.5 then it must also have python-simplejson package installed. 

SSH-key-based Authentication:

As ssh connection requires authentication each time it connects, hence the need for Key-based authentication is needed. Private and public key needs to be created and Public key needs to be pushed on to Managed hosts so that going forward we don't have to authenticate again and again. 
SSH key can be copied to different hosts with a command "ssh-copy-id" . 

Once ansible has been installed you can use the help option ( $ ansible -h ) to get help and ( $ ansible --version) to check for the installed version. 

Referencing Inventory Hosts:









Managing Ansible Configuration Files : 

Configuration of ansible can be controlled using the below files mentioned in the directories with Priorities.

1.  /etc/ansible/ansible.cfg --> This file is used when no other files are present. 
2.  ~/.ansible.cfg --> present in user's home directory. This is used b4 first entry if present. 
3.  ./ansible.cfg -->   If this file is present in the home directory from where the command is run then this will be used first.
4.  $ANSIBLE_CONFIG --> this is used to setup in an environment for Ansible if you have multiple locations to be run at multiple directories. This precedes all other entries. 

Due to these multitude of locations where ansible file can be placed, its very difficult to identify which files is being currently used by ansible, Hence in order to identify which file is currently being in use we can take help of the below command. 

# ansible --version 

 # ansible --version

   ansible 2.5.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules',       u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]


Another active way to see currently used ansible configuration is with -v option . 

# ansible server --list-hosts -v 

[root@Automation-1 ~]# ansible all --list-hosts -v
Using /etc/ansible/ansible.cfg as config file
  hosts (2):
    auto-2
    auto-3

Ansible configuration file breakup can be divided into below sections : 

[root@Automation-1 ~]# grep "^\[" /etc/ansible/ansible.cfg

[defaults]

[inventory]

[privilege_escalation]

[paramiko_connection]

[ssh_connection]

[persistent_connection]

[accelerate]

[selinux]

[colors]

[diff]

All the above mentioned parameters are configured ansible configurations but you have a miss here which is ( galaxy ) will talk about it later. 


1. Inventory -- > Location of the ansible inventory file. 
2. remote user -- > The remote user account used to establish connections to managed hosts. 
3. become -- >  Enables or disables privilege escalations for operations on managed hosts.
4. become_method -- > Defines the privilege escalations method on managed hosts. 
5. become_user -- > The user account to escalate privilege on managed hosts. 
6. become_ask_pass -- > Its for asking password while escalating user privilege on managed hosts.


Practicals : 

1. Log onto any system and run ansible version command to check active config files. 
#  ansible --version.

2. Open /etc/ansible/ansible.cfg file and examine the different sections for example. 
    Under the [default] section, locate and examine the inventory settings. 
    Check for [privilege_escalation] section. 
# cat /etc/ansible/ansible.cfg.


3. Create a user level Ansible configuration file at your home directory and then check which file is getting used for ansible config changes. 
# touch /home/xyz/ansible.cfg
# ansible --version.


4. Create a directory and when you are in that directory create a ansible cfg file and then check which file is being read for ansible configuration. 
# mkdir /home/xyz/vikas/ansible.cfg
# ansible --version. 


5. Create another user level ansible configuration file at /home/xyz/vikas/aaa/ansible.cfg. Set the $ANSIBLE_CONFIG  environment variable full path and then check which file is getting used currently. 
# touch /home/xyz/vikas/aaa/ansible.cfg. 
# export ANSIBLE_CONFIG=/home/xyz/vikas/aaa/ansible.cfg. 
# ansible --version. 

6. How to change default working inventory location for ansible. 
In the file ansible.cfg go to defaults section and modify the below : 

[defaults]
inventory=/home/xyz/vikas/inventory 

7. Go to that folder and create a hosts file in the directory. 
# cd /home/xyz/vikas/inventory
# vi hosts
   server1.lab.com
   server2.lab.com

Check with the below command if you are able to get the same content as below :
# ansible classroom --list-hosts.


Running AD-Hoc commands : 

- Running ad hoc commands locally.
- Running ad hoc commands remotely.
- Usage of ad hoc commands.


Ansible allows user to run/execute on-demand tasks on managed hosts. These ad hoc commands are the most basic operations that can be performed using ansible.

Each ad hoc command is capable of performing a single operation. If you need to run multiple commands then you will need to execute a series of ad hoc commands on the hosts.

Ad hoc commands are the simplest way for an user to start with learning ansible and then move on towards more complex ways like modules, plays and playbooks.

Alternatively ad hoc commands can be used to perform non-invasive commands such as querying a large group of managed for diagnostic information.

Syntax for Ad-hoc commands :

# ansible host-pattern -m module [ -a 'module arguments'] [-i inventory] 


Using modules in an Ad hoc commands: 

The modules are specified by -m options and this specifies which ansible module needs to be used to perform the remote operations. Will talk about modules later. 

Arguments are passed to a specified module using -a options. Some modules can handle no arguments and others can handle multiple arguments. When no arguments are needed simple run the command without providing -a option. if multiple arguments are needed then you can run them as below : 

# ansible host-pattern -m module -a 'argument1 argument2' [-i inventory]

Administrator has the rights of defining a default module that can be used by Ansible if no module has been specified. 
By default Ansible uses command module and mentioned in /etc/ansible/ansible.cfg file under 
# default module name for /usr/bin/ansible.
# module_name = command. 


So these 2 commands are equivalent to 1 another. 

Command 01 :  # ansible host-pattern -m command -a 'module arguments'
Command 02 :  # ansible host-pattern -a 'module arguments'


Mostly if you see below the normal Ansible output is fetched in 2 lines, 1 signifies the server it tried to reach and other is what it ran over there. 

You can get the Ansible output listed in 1 line with -o switch like below : 

[root@Automation-1 ansible]# ansible all -m command -a 'date' -o
auto-3 | SUCCESS | rc=0 | (stdout) Mon Aug  6 13:02:11 IST 2018
auto-2 | SUCCESS | rc=0 | (stdout) Mon Aug  6 13:02:02 IST 2018

Note : Drawbacks of command module over shell.

The commands run by using command module in Ansible cannot execute stuff related to piping and redirecting as they are not provided any shell by system when they execute. 

For such instance you need to use shell command instead of "command" command. 

Examples below : 

[root@Automation-1 ansible]# ansible auto-2 -m command -a set

auto-2 | FAILED | rc=2 >>

[Errno 2] No such file or directory

[root@Automation-1 ansible]# ansible auto-2 -m shell -a set

auto-2 | SUCCESS | rc=0 >>

BASH=/bin/sh

BASHOPTS=cmdhist:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath

BASH_ALIASES=()

Ad Hoc command configuration:
When an Ad Hoc command is executed, several things occur behind the scene. First the ansible configuration file is consulted for various parameters. Module_name that we had seen before is one such example. 

a. Connection settings: 
Reading the connection related parameters mentioned in the cfg file, Ansible triggers connection using the remote user mentioned in file. 

b. Privilege Escalation: 
After successfully connecting to a hosts, ansible can switch users before executing any operation.

Ansible Command line Options : 

Settings :                                                 Command line options 
Inventory                                                  -i 
remote_user                                            -u 
become                                                    --become, -b 
become_method                                      --become_method 
become_user                                           --become_user
become_ask_pass                                   --ask-become-pass, -k 

# ansible --help 

======================================================================
EXERCISE: 

1. [root@Automation-1 ~]# ansible auto-2 -m command -a 'id'
auto-2 | SUCCESS | rc=0 >>
uid=0(root) gid=0(root) groups=0(root)

2. Now we will try to change the remote file with dynamic content for ex: motd file.
[root@Automation-1 ~]# ansible auto-2 -m command -a 'cat /etc/motd'
auto-2 | SUCCESS | rc=0 >>


#  Here you can see that the file is empty .

3. Now lets try to put some content on to the empty file remotely.
[root@Automation-1 ~]# ansible auto-2 -m copy -a 'content="Managed by Vikas\n" dest=/etc/motd'
auto-2 | SUCCESS => {
    "changed": true,
    "checksum": "881a4e0ddbf6172bc0f7c4a0bb7919fbac59d6ab",
    "dest": "/etc/motd",
    "gid": 0,
    "group": "root",
    "md5sum": "ff7609058aa99d919ca273d99f2e1b95",
    "mode": "0644",
    "owner": "root",
    "size": 17,
    "src": "/root/.ansible/tmp/ansible-tmp-1534002527.61-15459222671883/source",
    "state": "file",
    "uid": 0
}

4. [root@Automation-1 ~]# ansible auto-2 -m command -a 'cat /etc/motd'
auto-2 | SUCCESS | rc=0 >>
Managed by Vikas

=================================================================

Managing Dynamic Inventory :

In this section we will use an Ansible Dynamic inventory to Pro-grammatically build an inventory from external data sources. 

By default, Ansible provides a text-based inventory format to define the hosts to be managed. When operating with larger systems ansible provides options of using the directories service monitored by them. Ansible supports dynamically building of an inventory from these external data sources through the use of scripts which retrieves information from them. 
AWS, Virtual environments also have such information of an instances such information can be used by ansible to build a hosts file in short time period. 

Difference between Static and Dynamic Inventory :
If the inventory file is executable then it is termed as dynamic inventory and if its not executable then its termed as static inventory. 

Supported Platforms : Below is the path of the scripts which can/will help you to generate a large number of inventory list if you have a large environment. 
Ansible Github site : https://github.com/ansible/ansible/tree/devel/contrib/inventory. 

1. Private cloud - redhat openstack platform. 
2. Public cloud - Rackspace, AWS & Google Space. 
3. Virtualization platforms like - OVIRT. 
4. Platform as a service solution - Openshift. 
5. Spacewalk. 

Writing Dynamic Inventory program : 

If a dynamic inventory script does not exists for the directory structure or infrastructure in use, It is possible to write a custom dynamic inventory program. Scripting can be done in any programming language, and it must return in JSON format when passed appropriate options. 

In order for an ansible script to retrieve list of hosts, script will have to run with option like --list parameter. Which in return should provide the details like group and hostname or IP address. 

# ./inventoryscript --list
{
        "Webservers" : [ "webserver1.example.com", "webserver2.example.com" ],
        "Database"   : [ "db1.example.com", "db2.example.com" ],
}

Working with multiple inventories:     

Ansible supports the use of Multiple inventories in the same run. If you place the inventory files in a directory and change the config files accordingly then all the inventory files under that directory will be read and executed. Path to change /etc/ansible/ansible.cfg. 

When multiple file exists in the same directory then they are examined in alphabetical order. In a similar fashion if you can include the inventory list so you can ignore them as per files. 

SUMMARY :

Lets summarize as to what we have completed until now: 

- Any system on which ansible is installed and which has access to right configuration files and playbooks to manage remote hosts can be termed as control hosts. 

- The managed hosts are defined in the inventory file. Host patterns are used to reference managed hosts defined in an inventory. 

- Inventory can be static or dynamic generated from a script or a program. 

- The location of the inventory should be managed by ansible.cfg file but it would great if its maintained at playbook directories. 

- Ansible look for a number of places for its configuration file. In an order mentioned before the first match point is taken by passing all the rest ansible cfg files. 

- The Ansible command is used to perform one time Ad-hoc requests on the server. 

- Ad Hoc commands determine the operation to perform through the use of modules and their arguments. 

- Ad Hoc commands which require additional permissions to get the job done, for these kind of jobs you can use escalation feature in ansible. 

Ansible Certification : 1. Introducing Ansible

1.   Introducing Ansible 

• Overview of Ansible Architecture. 
• Overview of Ansible Deployments.
• Describing Ansible Inventory
• Summary . 
• Quiz Details 
• Logs 

=======================================================================

INTRODUCING ANSIBLE

Ansible was originally written by Michael De Haan, the creator of the Cobbler provisioning application. Ansible is globally accepted because its easy to use and is built on Python. Ansible is also supported by Devops tools such as Vagrant and Jenkins. 

A file that contains a series of plays is called a playbook. Ansible is an open source configuration managment and orchestration utility. Ansible architecture is agentless. Work is pushed to remote hosts when Ansible executes. 

Modules are the actual programs which performs the actual work of the tasks in play. Ansible is immediately useful because it comes with hundreds of core modules that perform the system administrative tasks. 

What Ansible cannot do ? 

- Ansible cannot audit changes made on the system by other users. 

- Ansible can add packages to the system but it cannot add initial minimal installation of the systems. 

- Ansible can remediate system configuration file drift, it does not monitor it. 

- Ansible does not track what changes are made to the system from last deployment. 

ANSIBLE CONCEPTS AND ARCHITECTURE : 

2 types of machines in the ansible architecture. 

- The control node 

- The manged node 

Control node is where all the software resides. This is the command center for ansible. 

Ansible uses SSH as a network transport to communicate with the managed hosts. The module referenced in the playbook are copied to the managed hosts. Then they are executed in order, with the arguments specified in the playbook. 

Ansible control node components 

- Ansible Configuration : 

  Ansible has configuration setting which defines how it behaves. These settings include such as remote user + command execution. Providing password and sudo credentials while executing remote commands. Default configuration values can be overwritten by Env values and Values defined in Conf files. 

- Host Inventory : 

  The Ansible hosts inventory defines which configuration groups hosts belongs to. The inventory can define how Ansible communicates with the given hosts, 

- Core Modules  :

  Core modules are the modules that are shipped with Ansible. There are 400 core modules. 

- Custom Modules : 

   User can extend Ansible's Functionality by writing there own modules. Modules are typically written in Python, but user also has a option of writing the module in other languages like Perl, shell , Ruby etc. 

- Playbooks : 

   Ansible playbooks are files that are written in YAML Syntax that define the modules with arguments to apply with managed nodes. 

- Connection Plugins : 

   Plugins that enable communication with managed hosts or private cloud. These include native SSH, Parimiko ssh, and local. Parimiko is a python implementation of Openssh with RHEL6 that provides control persist setting to improve performance of Ansible. 

- Plugins : 

  Extensions that enhances Ansible's functionality. Examples emails, notification and logging. 

Roles and requirements for Control Node : 

Python 2.6 or Python 2.7 should be installed on control node. Configuration Files are maintained on Control node. 

Roles and requirements of Managed hosts: 

A managed hosts is a system into which ansible logs into and executes remote commands to perform configuration tasks. Ansible uses SSH so ssh must be configured to accept Nodes connections. Python-simplejson packaged needs to be installed on RHEL 5 version. Python 2.5 covers this package by default. 

QUIZ : 

1.  Which of the following programming language is Ansible built on ? 

• C ++
• Perl 
• Python 
• Ruby 

2.  Which of the following terms best describes Ansible's Architecture ?

• Agent-less.
• Client/Server 
• Event-Driven
• Stateless. 

3.  What is the network Protocol which Ansible uses to communicate with managed hosts ?

• HTTP 
• HTTPS
• Paramiko 
• SNMP 
• SSH 

4.   Which of the following files defines the action Ansible performs on Managed nodes ?

• Configuration Files. 
• Host Inventories. 
• Manifest 
• Playbooks. 
• Script. 

5.  What syntax is used to define Ansible Playbook ?

• Bash 
• Perl 
• Python 
• YAML 

Note : The serial keyword can be used to limit the number of hosts that the playbook runs at once. Once the subset of servers have been deployed and are functioning properly. Ansible will move onto another batch of server in the target group. By default, Ansible will try to apply playbook to the target managed hosts in parallel, with the exact number of parallel processes to spawn controlled by forks directive mentioned in the applicable ansible.cfg configuration files. 

Ansible Connections Plugins : 

Control Persist  : Connection plugins allow Ansible to communicate with managed hosts and cloud providers. The preferred connection plugins for newer version of Ansible is the native SSH options. Ansible uses control persist option when the client open-ssh supports it. 

Local Connection Plugin : Its used locally, mostly the use case for these types of scenarios are using a corn job to trigger a Ansible locally. 

Paramiko : It is used on RHEL 6. It's a connection solution for older systems where older version of open-ssh didn't had Control Persist. 

Winrm : Ansible connection plugin module allows Microsoft windows machines to be managed. The pywinrm needs to be installed on Linux machine to manage windows hosts. 

Docker connection plugin : Ansible 2 introduced docker as a plugin module which helps in communicating with the docker system without any SSH enabling on the client. 

QUIZ : 

1. Which of the following is not a deployment task suitable for Ansible ?

• - Deploy JBOSS consistently over different operating system. 
• - Deploy Red Hat Satellite agents to existing servers in Datacenter 
• - Discover the operating system version and software subscription status of RHEL. 
• - Monitor the state of the system so that it does not experience configuration Drift . 
• - Manage the software development life cycle of Openshift Enterprise Application. 

2. Which of the following Ansible Keyword facilitates zero-downtime rolling updates to occur by limiting the number of managed hosts a playbook can run on in parallel ?

- accelerate 

- gather_subset

- handlers

- serial 

- tasks. 

3. The paramiko Ansible connection plugin is used to communicate with which types of managed hosts ? 

- Docker containers.

- RHEL6

- RHEL7

- Windows Server


Describing Ansible Inventories: 

Ansible Inventories :
Ansible inventories are nothing but the host entries which ansible is going to manage. Hosts may belong to certain group which are identical to the pattern in Data-center. A host can be a member of more than one group.

2 types of host inventories :

• Static Inventory - its a text file .
• Dynamic Inventory - generated from outside providers. 

Static Host Inventories : 

An ansible static hosts is basically nothing but a txt file which is created to manage hosts and create groups for the ease of work. 

In the hosts file every server entry needs to be entered on a new line. You can put hostname or IP address. Host group needs to be defined within square brackets [ ]. 

Example : 2 groups are defined in the below hosts files.

# cat /etc/ansible/hosts
[webserver]
localhost
web1.example.com
web2.example.com:2233 ansible_connection=ssh ansible_user=goko
10.10.10.10

[database]
web1.example.com
db1.example.com

In the above example we can see that web2 is given directives that it has to use port 2233 and use ssh for connection and user used for login should be goko

Default location for hosts file :      /etc/ansible/hosts
Can be specified directly by using -i option or --inventory

2 groups can be clubbed together under a Parent group by using :children suffix.

Example as below :

# cat /etc/ansible/hosts
[auto]
hyundai
honda

[tyre]
mrf
apollo

[car:children]
auto
tyre

Now lets work on simplifying the work with hosts file if you have to deal multiple hosts which are in same range or have things in common.

Syntax : [start:end]

192.168.[4:6].[0:255]
server[1:10].example.com         --> server1 to 10 all are covered in here.

Now that we know most of the stuff on Ansible host inventories (static) let try some commands to help us find them at runtime.

$ ansible server1.example.com --list-hosts
---- > server1.example.com

$ ansible server01.example.com --list-hosts    -- > failed example.
---- > no output .

Defining variables in hosts files: 

Even though ansible allows you to specify the variables in hosts file but it is advisable to put them in specific directories.

Dynamic host inventory : 

This inventory can be dynamically generated. Source can be anything like cobbler, cloud, cmdb, cloud.

QUIZ: 

1. Which of the following items is not found in the Ansible inventory files ?

• Hosts group 
• IP Address range 
• Module names 
• Variable definitions 
• User authentication information. 

2. cat /etc/hosts
    [linux-dev]
    cchang.example.com
    rlocke.example.com

    [windows-dev]
    wdinyes.exmaple.com

    [development:children]
    linux-dev
    windows-dev

Given the ansible inventory above, which hosts groups include rlocke.example.com ?

• linux-dev
• windows-dev
• development
• both linux-dev & development.

3. Which of the following expressions can be used in an ansible inventory file to match hosts in the 10.1.0.0/16 address range ?

• 10.1.0.0/16
• 10.1.[0:255].[0:255]
• 10.1.[0-255].[0-255]
• 10.1*

4.  Which of the following can be a source for Ansible dynamic host inventory information ?

• Cobbler system information 
• Configuration management system 
• LDAP Database
• Scripts that fetch information from Cloud 
• All of the above. 

QUIZ : 

1. The python-simplejson package must be installed on which of these nodes ?

• Ansible control node. 
• RHEL 5 
• RHEL 6 
• RHEL 7 
• Windows managed hosts 

2.  What is the default location of Ansible hosts file ?

• /etc/ansible/inventory
• /etc/ansible/hosts
• /etc/ansible/hosts.groups
• /etc/ansible/hosts.inventory. 

Ansible Certification [EX407] - Index Page

Introduction 

• Automation with Ansible. 

1.   Introducing Ansible 

• Overview of Ansible Architecture. 
• Overview of Ansible Deployments.
• Describing Ansible Inventory
• Summary . 
• Quiz Details 
• Logs 

2. Deploying Ansible. 

• Installing Ansible.
• Managing Ansible Configuration Files.
• Running Ad Hoc Commands.
• Managing Dynamic Inventories.
• Summary
• Lab:
• Exercise

3.   Implementing Playbooks

• Writing YAML Files
• Implementing Modules.
• Implementing Ansible Playbooks
• Summary
• Lab
• Exercise

4. Managing Variables and Inclusions

• Managing Variables
• Managing Facts
• Managing Inclusions
• Summary
• Lab
• Exercise

5.    Implementing Task Control.

• Constructing Flow Control
• Implementing Handlers.
• Implementing Tags
• Handling Errors
• Lab
• Summary

6.     Implementing Jinja2 Templates.

• Describing Jinja2 Templates.
• Implementing Jinja2 Templates.
• Summary
• Lab
• Exercise

7.     Implementing Roles.

• Describing Role structure
• Creating Roles.
• Deploying Roles with Ansible Galaxy
• Summary
• Lab
• Exercise

8.    Optimizing Ansible

• Configuring Connection Types.
• Configuring Delegation
• Configuring Parallelism
• Summary
• Lab
• Exercise.

9.    Implementing Ansible Vault.

• Configuring Ansible Vault
• Executing with Ansible Vault
• Summary
• Lab
• Exercise

10.    Troubleshooting Ansible

• Troubleshooting Playbooks
• Troubleshooting Ansible Managed Hosts
• Summary
• Lab
• Exercise

11.   Implementing Ansible Tower

• Describing Ansible Tower
• Deploying Ansible Tower
• Configuring Users in Ansible
• Managing Hosts in Ansible Tower
• Managing Jobs in Ansible Tower
• Summary
• Lab
• Exercise

12. Implementing Ansible in Devops Environment

• Provisioning Vagrant Machines
• Deploying Vagrant in a DevOps Environment.
• Summary
• Lab
• Exercise

13.   Automation with Ansible

• Review
• Labs

Unknown Files

Files with no inode, no owner and no group.

 Some of files in one of partition formatted with ext3 was showing ? in place of inode, owner and group position. Any Ideas why ? After doing a lot of research it can be said that it was as stated below

root# cd /tmp/vikas
root# ls -li
? ? ? ? ? abc.txt

I become confused. find command also report a number of files which has no owner
root# find /tmp/vikas -nouser

Investing through problem result in conclusion that there is file system error in that partition. I advised apply fsck  command immediately. Applying fsck solved the problem but i am searching in what condition file exist without inode number.

root# fsck -y /dev/vg1/lv1

I also ensured that that file system get applied fsck on  reboot

root# shutdown -Fr now
                                       One interesting point i learned in between is that maximum 16 consequent times a file system can get mounted without applying fsck after that a warning come to fsck although this setting can be override with tune2fs command.

  The best option to reduce file system error is to apply fsck at booting time. This can be easily done by making entry in /etc/fstab for example in my case the entry is

/dev/vg1/lv1  /data  ext3  defaults   1   2

Here 1 says apply fsck and 2 says after applying fsck on root.

FSCK AND SUPER-BLOCK

SUPER-BLOCK Recovery and FSCK Stages:

 

As we know fsck  is a great command to check and repair error on file system. Many times i found  filesystem in panic and fsck make it operation-able.  I used fsck many times but every time i ensured that filesystem on which i applying fsck be in unmount state.
But  the most important thing i was interested is fsck stages. Whenever i issued fsck command it output as

   Phase 1: Checking Inodes,blocks and sizes
   Phase 2: Checking directory structure
   Phase 3: Checking directory connectivity
   Phase 4: Checking Reference count
   Phase 5: Checking group summary information

fsck checks the integrity of several different features of the file system. Most important checking that fsck do is of super block. As we know super block is most important aspect of file system which stores summary information for the volume. super block is also most modified item in file system so chances of corruption of super block is always high.
Checks on the superblock include:

• A check of the file system size, which obviously must be greater than the size computed from the number of blocks identified in the superblock
  
• The total number of inodes, which must be less than the maximum number of inodes
  
• A tally of reported free blocks and inodes

On number of occasion I found super block of my file system get corruption. Although its a very difficult for me to dictates reasons of super block corruption. But the better part is that a backup super block is always present in our file system. To know that where is alternate super block we can use dumpe2fs command as follwing

    root# dumpe2fs /dev/sda1|more

Generally block number 32768 is backup super block, so to recover filesystem using backup super block fsck command can be used in following ways

 root# fsck -b 32768 /dev/sda1

 Other than super block corruption other error can be easily fixed by using  fsck in straight ways as following

 root# fsck -y /dev/sda1 (Here -y option save us from pressing y while asking for yes during recovery )

When inodes are examined by fsck, the process is sequential in nature and aims to identify inconsistencies in format and type, link count, duplicate blocks, bad block numbers, and inode size. Inodes should always be in one of three states: allocated (being used by a file), unallocated (not being used by a file), and partially allocated, meaning that during an allocation or unallocation procedure, data has been left behind that should have been deleted or completed.

 Alternatively, partial allocation could result from a physical hardware failure. In both of these cases, fsck will attempt to clear the inode. The link count is the number of directory entries that are linked to a particular inode. fsck always checks that the number of directory entries listed is correct, by examining the entire directory structure beginning with the root directory, and tallying the number of links for every inode. Clearly, the stored link count and the actual link count should agree, but the stored link count can occasionally be different than the actual link count.

This could result from a disk not being synchronized before a shutdown, for example, and while changes to the file system have been saved, the link count has not been correctly updated. If the stored count is not zero, but the actual count is zero, then disconnected files are placed in the lost+found directory found in the top level of the file system concerned. In other cases, the actual count replaces the stored count.

Conga Cluster Commands

Redhat Cluster Commands at one go 

Here I will try to cover all the basic commands that are required to configure and work on conga cluster. The motive behind this is to remember all the related commands at one go..

For detailed information I would be writing more on the same front latter down the line... Let me know if you are interested.

 ----------------------------------------------------------------------------------------------------------

# luci_admin init
# service luci restart
# luci_admin passwd

# lvm dumpconfig | grep locking type
# lvmconf --enable -cluster
# lvm dumpconfig | grep locking type

# ip addr list
# clustat
# clustat -i 1
# css_tool addnodeid
# cman_tool status
# cman_tool nodes
# cman_tool join -p "id"
# css_tool update /etc/cluster/cluster.conf
# fence_tool join -w
# gfs_fsck
# fenced_node node1
# cman_tool kill -n node1
# service qdisk start
# mkqdisk -C device -l label
# mkqdisk -l
# clulog
# cman_tool leave|remove
# cman_tool votes -v
# cman_tool expected -e

Umask explained in Linux

UMASK in linux not so confusing any more

What is UMASK and how to define it in Linux?

UMASK(User Mask or User file creation MASK) is the default permission or base permissions given when a new file(even folder too, as Linux treats everything as files) is created on a Linux machine. Most of the Linux distros give 022(0022) as default UMASK. In other words, It is a system default permissions for newly created files/folders in the machine.

How to calculate UMASK in Linux?

Though umask value is same for files and folders but calculation of File base permissions and Directory base permissions are different.

The minimum and maximum UMASK value for a folder is 000 and 777
The minimum and maximum UMASK value for a file is 000 and 666

Why 666 is the maximum value for file?

This is because only scripts and binaries should have execute permissions, normal and regular files should have just read and write permissions. Directories require execute permissions for viewing the contents in it, so they can have 777 as permissions.
Below are the permissions and its values used by UMASK. If you are Linux/Unix user you observe these are inverse to actual permissions values when setting up permissions to files/folders with CHMOD command.

 0 --Full permissions(Read, Write, Execute)
 1 --Write and read
 2 --Read and execute
 3 --Read only
 4 --Write and execute
 5 --Write only
 6 --Execute onlyadminadmin
 7 --No permissions

How to remember these and calculate the file and folder permissions?

Consider above values are inverse to actual permissions. Suppose your UMASK value is 0027(027).

For folder:
To calculate actual folder permissions from UMASK is done in two steps

Step1:Logical Negate the UMASK

Not(027) = 750

Step2: Logical AND this number with 777

777 AND 750 = 750

So actual folder permissions is 750 when its created. Owner will get full permission, group gets execute and write permissions and others no permissions
In other words and simple way..
We have to subtract 027 from 777 then we will get the actual folder permissions.

777 - 027 = 750

which is nothing but full permissions for the owner, read and execute permissions for group and no permissions for others.
For files:
To get actuall file permissions from UMASK is done in two steps

Step1:Logical Negate the UMASK

Not(027) = 750

Step2: Logical AND this number with 666

666 AND 750 = 640

For your understanding purpose we have calculated this below equation to get what actual AND operator do.

110 + 111 = 110(6)
110 + 101 = 100(4)
110 + 000 = 000(0)

How to see default UMASK?

just type umask and you will get whats the default UMASK

umask

Output

0022

Questions and Answers related to UMASK

1)How to setup or change default UMASK for all the new users?

The UMASK value can be set in /etc/profile for all the new users. Open this file as root user and given the below line in the file.

umask 027

2)How to setup or change default UMASK for existing users?

For existing users you can edit ~/.bashrc file in their home directory. This should be done for all the users one by one or if the machine is having lots and lots of users then you can write a shell script for this.

3)I see people are using 0022 and 022 as UMASK, is there any difference between them?

There is no difference between these two, both indicates one and the same. The preceding 0 indicates there is no SUID/SGID/Sticky bit information set.

4)What is the perferred UMASK value for a system for Security reasons?

Prefered is 027(0027) for security reasons becasue this will restrict others not to read/write/execute that file/folder

5)I see umask value as 022 in my vsftpd config file? what actually this means?

When you see 022 as umask value in vsftpd config file that indicates that users who are going to create files will get 644  and for folders its 755 respectively.
To know more about umask refer man pages and info pages.

man umask
info umask

YUM Server in RHEL 5 configuration

How To Install YUM Server In Red-hat(RHEL5)

In Linux you can install packages through many ways like.
1. Through RPM,
2. Through shell script
3. Through source tar balls etc.

YUM(Yellow-dog Updater and Modifier) is another and advanced way of installing the packages in Linux distro’s such as Red-hat, Fedora and CenOS.
In RHEL4 installing packages is a tedious process, some times its headache to install all the dependencies. So Red-hat come with a solution to overcome this dependencies problem in most situations, the solution for this is nothing but YUM implementation. This will resolve this dependency issue and other known issues. Here we are going to present some basic way how to use YUM utility to install packages locally(there are so many ways to install packages from different sources either local or remote such as ftp, http).

Basic YUM implementation locally:

Step1 :Copy the entire OS cd’s (DVD) content to Hard-drive as below.

#cp -ar /media/cdrom/Server/* /destinationfolder

Example :

server1#cp -ar /media/cdrom/Server/* /var/ftp/pub/Server/

Note :
1. From second cd too Server content in to our /var/ftp/pub/Server/ folder as shown below.
2. Here please take destination folder as /var/ftp/pub so that we can implement FTP server to share our repository.
3. If you have RHEL5 DVD then execute below command once.

server1#cp -ar /media/cdrom/Server/* /var/ftp/pub/Server/

Step2 : Now change the directory to /var/ftp/pub/Server and install the createrepo package

server1#cd /var/ftp/pub/Server
server1#rpm -ivh createrepo*   

Step3 : Specify the repository location to YUM.

server1#createrep -v .

Note:There is the dot in the above command.


Step4 : Create a file with repo as extension and specify the YUM details in /etc/yum.repos.d folder

server1#cd /etc/yum.repos.d/
server1#vi testing.repo

Note : The directory /etc/yum.repos.d/ contaions two .repo files which should be removed or moved to other directory, so that YUM server will check default .repo file it self.

Q. What if I don’t move or remove the default .repo files from /etc/yum.repos.d/ folder?
A: Every time when you try to install packages through YUM, your yum will check all these files for repositories for getting packages which will or will not work and most probably delay is increased in getting those packages from online servers.
The new file which is created contains as follows.

[server1.example.com]
comment ="Vikas Linux Repo -- Repo 01"
baseurl=file:///var/ftp/pub/Server
gpgcheck=0

After entering these entries save and exit from the file.
Let me explain what actually these four entries mean.
[server1.example.com] ==>This informs what is the repository name.
comment ==> Its used to see the information about the repo.
baseurl ==> This is the server and path of the repo(here its a local repo so the base url is just a file:///
For example you are creating YUM client through FTP then base url should be like this
baseurl=ftp://station1.example.com/pub/Server
gpgcheck ==> This is to check the authentication of the repository, which is disabled in this case.
Local YUM repository is created, now you can install any package you want with yum command. In order to know more about YUM, Please see man pages for YUM.
Note:In-order to use yum repository we have to clean the yum meta data, so before installing any package first time use yum clean all command as shown below.

server1#yum clean all

Basic YUM Server Repository through FTP server :
So what about Installing packages remotely by using this repository?
Let us see how to configure client to access this repository. Before doing client configuration we have to share this repository through FTP or HTTP.

Step1 : Install vsftpd server on server

server1#yum install vsftpd

Step2 : Start the ftp service and on it

server1#service vsftpd restart
server1#chkconfig vsftpd on

That’s it on the server side every thing configured properly, Now move on to client machine.

Conflagration on client side :
Step3 : Remove/move the local repository file from /etc/yum.repo.d/ folder to some other location #mv /etc/yum.repo.d/* /tmp/
Step4 : Create server.repo file in /etc/yum.repo.d/ with following contents

[server1.linuxnix.com]

comment ="test"

baseurl=ftp://server1.example.com/pub/Server

gpgcheck=0

Save and exit the file Now start using yum to install packages, as follows.

client1#yum clean all
client1#yum install packagename

Example :

client1#yum install httpd

To uninstall a package through YUM

client1#yum remove httpd

To see the info of a package

client1#yum info packagename

To see the package is already installed or not

client1#rpm -qa grep packagename