FUNC - Fedora Unified Network Controller

INTRODUCTION

FUNC is an open source automation tool developed by using Python programming language. We can use this tool for automating system admin tasks such as status check, configuration tweak, file transfer, rebooting the systems etc in multiple Linux-based systems. It uses the typical Client-Server model, where the server is called by the term "overlord" and the clients which are binded to the server are called as "minions".

INSTALLATION PROCEDURE

The installation procedure for Server and Clients are same but the configuration makes the difference to run as a Server or Client. 

Following are the required packages which have to be installed. 

smolt

python-ctypes

python-paste

python-simplejson

pyOpenSSL 

certmaster
func

The main configuration file for Server is "/etc/certmaster/certmaster.conf" and for client is "/etc/certmaster/minion.conf"
For a Server to run it requires a service by name "certmaster" and it listens to port 51235.
For a Client to run it requires a service by name "funcd" and it listens to port 51234.

FUNC Client installation on 32bit OS

scp the following file to local system and install the RPM in same order as mentioned.

pyOpenSSL-0.6-2.el5.i386.rpm

certmaster-0.27-1.el5.noarch.rpm
python-ctypes-1.0.2-2.el5.i386.rpm
python-paste-1.7.4-1.el5.noarch.rpm

python-simplejson-2.0.9-8.el5.i386.rpm

smolt-1.4-2.3.el5.noarch.rpm

func-0.27-1.el5.noarch.rpm

Configuration
#edit /etc/certmaster/minion.conf file with following content

# configuration for minions
[main]
certmaster = <FUNC Server IP>
certmaster_port = 51235
log_level = DEBUG
cert_dir = /etc/pki/certmaster

start the services
#service funcd start
#chkconfig funcd on 

Note: We should ensure the Network team allows TCP traffic through on these ports.   

HOW IT WORKS  

Func uses an X.509 public key infrastructure (PKI) to authenticate the minions to the overlord daemon (Master). These SSL certificates are also used to encrypt and verify all communications.

The certmaster daemon running on the overlord creates a Certificate Authority (CA) when it is first started. This serves as the root of the PKI. As each minion starts it will generate a key pair and a Certificate Signing Request (CSR) which contains its public key. The CSR is uploaded to the Master. After the minion has its certificate signed by the overlord's CA you can start issuing commands to that host.

Keys are stored in /etc/pki/func for both sides. Certmaster stores certs and csrs in /var/lib/func (and do not need to be protected against unauthorized read access).

Note: Despite the fact that we have a SSL communication established between Server and clients, we still can't login to client server (from Master) without a password. We can issue commands to client machines only using Func module. Hence this prevents the security risk like we have in establishing SSH connection using empty phase phrase.

HOW TO USE

The command-line syntax for Func is as follows: 

# func "hostnames" module method [arg1] [arg2] 

Every command should start with the keyword 'func' followed by 'hostnames' within double quotes, module name, method name and the arguments. 

In Detail: 

"Hostname" - Hostnames within double quotes can be a single servername or a servernames separated by semicolons or we can use wildcard characters to specified a set of servers. 

"Module" - In a simple term, we can say a module is a high-level description of an entity and it contains number of useful functions or a set of commands. In OOPS term, we can call it as a Class which contains number of methods. 

"Methods" - These are functions or a set of command set available under a particular Module 

"Arguments" - Parameters which are passed to methods. 

Note: In Func, we have an option of creating Python API which does the same as we do in command-line and include it in a Python script. But I haven't covered it in this document. 

EXAMPLES 

To list the minions connected to the Master server:

[root@bkpllm01 ~]# func "" list_minions*
 
dbsllm01-d1
webllm04-t1
webllm04-t2
webllm04-t3

To obtain the list of all the remote modules we can use on a particular client server (here it is 'applsb01')

[root@sysllm01 ~]# func "applsb01" call system list_modules
{'applsb01': ['bridge',
              'certmastermod',
              'command',
              'copyfile',
              'delegation',
              'djangoctl',
              'echo',
              'filetracker',
              'func_getargs',
              'func_module',
              'hardware',
              'iptables',
              'iptables.port',
              'jboss',
              'jobs',
              'meta',
              'mount',
              'nagios-check',
              'netapp.options',
              'netapp.snap',
              'netapp.vol',
              'netapp.vol.clone',
              'networktest',
              'overlord',
              'process',
              'reboot',
              'rpms',
              'service',
              'smart',
              'snmp',
              'sysctl',
              'test',
              'vlan',
              'yumcmd']}

Here is the example for how the Method in each module can be used listed and used in real time

To obtain the list of methods offered by a particular module (say jboss), the syntax would be:

[root@sysllm01 ~]#  func "applsb01" call jboss list_methods
{'applsb01': ['status',
              'save_config',
              'search_by_port',
              'search_by_address',
              'search_by_instance',
              'check',
              'module_description',
              'module_version',
              'get_method_args',
              'module_api_version',
              'list_methods']}

[root@sysllm01 ~]#  func "applsb01" call jboss status
{'applsb01': [[31253, 'ihcore', 'applsb01', []]]}

To run the 'uptime' command in all the minions

[root@sysllm01 ~]# func "appls*" call command run uptime
('applsb01',
 [0,
  ' 11:20:24 up 120 days, 20:10,  2 users,  load average: 0.00, 0.00, 0.00\n',
  ''])
('applsb02',
 [0,
  ' 11:20:25 up 120 days, 12:08,  1 user,  load average: 0.01, 0.05, 0.01\n',
  ''])
('applsb03',
 [0,
  ' 11:20:24 up 179 days, 13:04,  1 user,  load average: 0.03, 0.01, 0.00\n',
  ''])
('applsb04',
 [0,
  ' 11:20:25 up 120 days,  4:03,  2 users,  load average: 0.14, 0.05, 0.01\n',
  ''])

Here is an another example of using command module to execute remote commands

[root@sysllm01 ~]# func "appls*" call command run "uname -a"
('applsb01',
 [0,
  'Linux applsb01 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux\n',
  ''])
('applsb02',
 [0,
  'Linux applsb02 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux\n',
  ''])
('applsb03',
 [0,
  'Linux applsb03 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux\n',
  ''])
('applsb04',
 [0,
  'Linux applsb04 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux\n',
  ''])

More on JBoss module

This module allow you to monitor and control your jboss instances; it permit to get status of instances (listening port, pid, instance name and bind address), simple search by address, by port and by instance name, and check instances problems (example when a jboss instance is up but don't listen on any ports).

Command Line Usage

func "*" call jboss status
func "*" call jboss check
func "*" call jboss search_by_instance "default"
func "*" call jboss search_by_address "127.0.0.1"
func "*" call jboss search_by_port 8080
func "*" call jboss start
func "*" call jboss stop

Details start(), stop() need a proper configuration. The default values are

jboss_home=/var/lib/jboss
jboss_instance=default
jboss_address=127.0.0.1

If you have different settings, you must create and edit a configuration module file. Your file is /etc/func/modules/JBoss.conf and should be configure (for example) as below:

[main]
jboss_home=/usr/local/jboss-4.2.2.GA/
jboss_address=127.0.0.1
jboss_instance=default

status() return, for each host a list with pid, instance name, bind address and a list of listening ports:

[root@sysllm01 ~]#  func "applsb01" call jboss status
{'applsb01': [[31253, 'ihcore', 'applsb01', []]]}

Service Module

It starts, stops, and checks the status of services.
Command line usage:

[root@sysllm01 ~]# func applsb01 call service status vsftpd
{'applsb01': 0}
0--Success

[root@sysllm01 ~]# func applsb04 call service status vsftpd
{'applsb04': 3}

3-- Other than 0 in the output meant for Failure or the service doesn't exist on remote system

SUMMARY

Func is a great automation tool, invented exclusively for performing system admin tasks more effective than before on Linux platform. It totally eliminates the difficulties in login to multiple servers without compromising the security and it's pretty faster in executing the commands in remote clients. In addition to taking care of client login issue, it makes our work easier by capturing the result of command execution on remote machines and displaying it locally on Server's terminal. As we know, capturing the output of remote execution is indeed a major challenge and Func comes as a handy tool by eliminating it. Having said all this, Func does have some limitations. First is, the Func output is somewhat chaotic and we can't expect a formatted output. Next is, we can't do every system admin tasks using Func and it's not meant to replace the scripting languages such as Perl, Bash, Expect etc. Instead we can use the Func as a component in our scripting and derive some logic out of it. All we have to do is to learn the functionalities of Func modules/methods and see how we can use in our environment.

  

Information Source https://fedorahosted.org/func/