Amazon New features! Loan balancing and monitoring.... finally!

Dear AWS Community Member,

You signed up to be notified when we released monitoring, auto scaling and load balancing for Amazon EC2. We are excited to announce the public beta of these new features: Amazon CloudWatch, a web service for monitoring AWS cloud resources, Auto Scaling for automatically growing and shrinking Amazon EC2 capacity based on demand, and Elastic Load Balancing for distributing incoming traffic across Amazon EC2 compute instances. Together, these capabilities provide you with visibility into the health and usage of your AWS compute resources, enhance application performance, and lower costs.

Monitoring

Amazon CloudWatch is a web service that provides monitoring for AWS cloud resources, starting with Amazon EC2. It provides customers with visibility into resource utilization, operational performance, and overall demand patterns -- including metrics such as CPU utilization, disk reads and writes, and network traffic. To use Amazon CloudWatch, simply select the Amazon EC2 instances that you'd like to monitor; within minutes, Amazon CloudWatch will begin aggregating and storing monitoring data that can be accessed using web service APIs or Command Line Tools.

Auto Scaling

Auto Scaling allows you to automatically scale your Amazon EC2 capacity up or down according to conditions you define. With Auto Scaling, you can ensure that the number of Amazon EC2 instances you're using scales up seamlessly during demand spikes to maintain performance, and scales down automatically during demand lulls to minimize costs. Auto Scaling is particularly well suited for applications that experience hourly, daily, or weekly variability in usage. Auto Scaling is enabled by Amazon CloudWatch and available at no additional charge beyond Amazon CloudWatch fees.

Elastic Load Balancing

Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances. It enables you to achieve even greater fault tolerance in your applications, seamlessly providing the amount of load balancing capacity needed in response to incoming application traffic. Elastic Load Balancing detects unhealthy instances within a pool and automatically reroutes traffic to healthy instances until the unhealthy instances have been restored. Customers can enable Elastic Load Balancing within a single Availability Zone or across multiple zones for even more consistent application performance.

Like all Amazon Web Services and features, Amazon CloudWatch and Elastic Load Balancing are available on a pay-as-you-go basis with no up-front fee, minimum spend or long term commitment. Auto Scaling is free to Amazon CloudWatch customers. Each instance launched by Auto Scaling is automatically enabled for monitoring and the Amazon CloudWatch monitoring charge will be applied.

For more information on these new features and details on how to start using them, please see the resources listed below:

Amazon EC2 Detail Page

Release Notes

These have been among the most requested Amazon EC2 features by our customers. We hope they prove useful to you, and we look forward to your feedback.

Sincerely,

The Amazon Web Services Team

Amazon has a new feature, reserved instances.

Basically you pay a one time up front fee, and it drastically lowers your hourly $ cost per instance.

You can have a small instance (Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform)

For around - 65$/month - 780$/year, you could have a dedicated server ... pretty sweet deal.

Now if you add in the reserved instance's feature...

A small instance will cost
$325(reserved instance fee) + $262/year = $587/year ... even better deal.

Sign up for 3 years, and you now are paying...
$587 1st year, and then $262 ... end of 3 years total = $1111 compared to $2340 for the regular price for 3 years.

Rebundeling a running ec2 instance....

ec2-bundle-vol --prefix what_you_want_to_name_it -d /mnt/ami -c pathtocert.pem -k pathtokeyfile.pem -u 123456789 -s 10240 --kernel aki-9b00e5f2 -r i386

-d, --destination PATH
-c, --cert PATH
-k, --privatekey PATH
--kernel ID Id of the default kernel to launch the AMI with.
-r, --arch ARCHITECTURE Specify target architecture. One of ["i386", "x86_64"]
-s, --size MB The size, in MB (1024 * 1024 bytes), of the image file to create. The maximum size is 10240 MB.

(change the kernel type and arch to suite your needs... might as well use the largest size 10240)

This will bundle your running instance, and place the files in /mnt/ami

ec2-upload-bundle -b bucketname -m /mnt/ami/what_you_named_it.manifest.xml --access-key XYZ --secret-key XYZ

This will upload your bundled image to your bucket.

ec2-register /bucket/what_you_named_it.manifest.xml

The last step is to register your image, you will get back the AMI ID, and can either start your instance on the cmd line, or simply login to the web console and start it.

Nagios - Instead of a ping check to see if a host is alive, use http instead

Sometimes (often) ICMP is blocked, so you can't ping check your hosts to see if they are alive.

You can add this to commands.cfg and hosts.cfg for those hosts to check via http instead.

commands.cfg

define command {
   command_name check-host-alive-by-http
   command_line $USER1$/check_http -H $HOSTADDRESS
}

hosts.cfg

define host{
host_name hostname.com
address ip.address
alias hostnamealias
use networking_machines_template
check_command check-host-alive-by-http
}

Installing Nagios

Installing Nagios
---------------------
*need to have basic centos install, with apache installed.



Install rpmforge repo

wget http://apt.sw.be/redhat/el5/en/i386/RPMS.dag/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
wget http://apt.sw.be/redhat/el5/en/x86_64/RPMS.dag/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
rpm --import http://dag.wieers.com/rpm/packages/RPM-GPG-KEY.dag.txt
rpm -K rpmforge-release-0.3.6-1.el5.rf.*.rpm #verify package
rpm -i rpmforge-release-0.3.6-1.el5.rf.*.rpm #install package

Install yum-priorities #this is't required, but I like to limit the rpmforge repository from effecting any base packages

yum install yum-priorities
#Make sure that yum-priorities is enabled by editing the /etc/yum/pluginconf.d/priorities.conf
#Edit the .repo files in /etc/yum.repos.d/ and set up priorities by adding the line: (lower number = higher priority, 0 = disabled)
priority=N


Install nagios packages

sudo yum install nagios nagios-devel nagios-plugins nagios-plugins-setuid rrdtool


Configure Nagios

sudo htpasswd -c /etc/nagios/htpasswd.users kylec #Create htpasswd file for auth
sudo htpasswd /etc/nagios/htpasswd.users username #for adding users

sudo vi /etc/nagios/nagios.cfg
comment out...
#cfg_file=/etc/nagios/objects/templates.cfg
#cfg_file=/etc/nagios/objects/localhost.cfg
add...
cfg_file=/etc/nagios/objects/hosts.cfg
cfg_file=/etc/nagios/objects/hostgroups.cfg
cfg_file=/etc/nagios/objects/services.cfg
cfg_file=/etc/nagios/objects/contactgroups.cfg
set...
process_performance_data=1
host_perfdata_command=process-host-perfdata
service_perfdata_command=process-service-perfdata


sudo vi /etc/nagios/cgi.cfg
set...
authorized_for_system_information=*
authorized_for_configuration_information=*
authorized_for_system_commands=*
authorized_for_all_services=*
authorized_for_all_hosts=*
authorized_for_all_service_commands=*
authorized_for_all_host_commands=*


----------------------------------------------------------------------------------------------
Installing Nagios pnp

wget http://switch.dl.sourceforge.net/sourceforge/pnp4nagios/pnp-0.4.13.tar.gz
./configure
make all
make fullinstall

edit hosts.cfg

add:

define host{
use generic-host,host-pnp #add host-pnp to networking_machines_template


define host {
name host-pnp
register 0
action_url /nagios/pnp/index.php?host=$HOSTNAME$' onmouseover="get_g('$HOSTNAME$','_HOST_')" onmouseout="clear_g()"

}

edit services.cfg

Add to the main template... (in my case basic-service)

define service{
use generic-service,srv-pnp (srv-pnp is whats added)
name basic-service
.... truncated


define service {
name srv-pnp
register 0
action_url /nagios/pnp/index.php?host=$HOSTNAME$&srv=$SERVICEDESC$' onmouseover="get_g('$HOSTNAME$','$SERVICEDESC$')" onmouseout="clear_g()"

}

edit commands.cfg

add:

define command {
command_name process-service-perfdata
command_line /usr/bin/perl /usr/local/nagios/libexec/process_perfdata.pl
}

define command {
command_name process-host-perfdata
command_line /usr/bin/perl /usr/local/nagios/libexec/process_perfdata.pl -d HOSTPERFDATA
}

comment out the existing SAMPLE PERFORMANCE DATA COMMANDS

setup configs

cd /usr/local/nagios/etc/pnp/
sudo mv npcd.cfg-sample npcd.cfg
sudo mv process_perfdata.cfg-sample process_perfdata.cfg
sudo mv rra.cfg-sample rra.cfg


If you are seeing ... "File does not exist: /usr/share/nagios/pnp," in your httpd logs.
cp -R /usr/local/nagios/share/pnp/ /usr/share/nagios/pnp

edit as you would like them.

start npcd
/etc/init.d/npcd start
-------------------------------------------------------------------------------------------------------




EXAMPLE .cfgs #this is custom, these are just some examples to get you started.

contacts.cfg

define contact{
contact_name kyle
alias kyle
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,u,r
service_notification_commands notify-by-email
host_notification_commands host-notify-by-email
email kyle@email.com
}

contactgroups.cfg

define contactgroup{
contactgroup_name admins
alias admins
members internal_infosec, kylepager
}


hosts.cfg

define host{
host_name website.com
address website.com
alias website.com
use networking_machines_template
}


hostgroups.cfg

define hostgroup{
hostgroup_name webservers
alias webservers
members server1, server2, server3, website.com
}

services.cfg

define service{
use basic-service
hostgroup_name webservers
service_description HTTP
check_command check_http!
contact_groups admins
}

Use nmap to scan for conflicker...

Use nmap to scan for conflicker...
http://seclists.org/nmap-dev/2009/q1/0869.html

get the latest of nmap, and install lua

Directions for ubuntu ...

sudo apt-get install lua50

wget http://nmap.org/dist/nmap-4.85BETA7.tar.bz2
tar -jzxvf nmap-4.85BETA7.tar.bz2
./configure
make

./nmap -sC --script=smb-check-vulns --script-args=safe=1 -p445 -d -PN -n -T4 --min-hostgroup 256 --min-parallelism 64 -oA conficker_scan 192.168.1.1/24 | grep Conficker:


You should see all

| Conficker: Likely CLEAN

Just remove the grep filter to see the host if you get any other results.

Modified pidgin SMS for added functionality

This is by no means very well written. But its a short easy hack to get IM's sent to you via SMS, that is somewhat dynamic so you can receive them from anyone you have it setup for.

1.) Select buddy to add pounce
2.) Select your bounce options
3.) Select it to execute your script, and after the script path, add the full buddy name (this will be the path to the log that is the variable part)


The only part I don't have very dynamic is this will only work the way its intended if your buddy's are all under the same account, you need to have a different script for each account you use.

pidgin.sms.sh
------------------------------------------------------------------------------------------------------
#/bin/bash
#Written by kylepike
#This script will send you an sms txt alert if you have a buddy pounce setup in pidgn
#The argument you need to pass in is the full "buddy name" in the "New buddy pounce" window


buddyfolder="/home/kyle/.purple/logs/jabber/kyle@k0rupted.domain.net/$1"

cd $buddyfolder

file=`ls -lrt | tail -n 1 | awk '{ print $8}'`
lynx --dump $file | tail -n 1 > ~/scripts/emailmessage.txt
scp ~/scripts/emailmessage.txt kyle@k0rupted.domain.net:~/scripts/emailmessage.txt

ssh kyle@k0rupted.domain.net /home/kyle/scripts/smsme.sh
-------------------------------------------------------------------------------------------------------


smsme.sh on the remote sendmail server, you could incorporate this into one if you have a working sendmail server on your computer/laptop. But like I said in the earlier post, I'm always roaming, and paranoid and I would't want to send my sms out from an unsecure network clear txt.

#!/bin/bash
# script to send simple email
# email subject
sn="SenderName"
SUBJECT="IM FROM $sn"
# Email To ?
EMAIL="123456789@messaging.sprintpcs.com"
# Email text/message
EMAILMESSAGE="/home/kyle/scripts/emailmessage.txt"
#echo "IM Reminder"> $EMAILMESSAGE
#echo "From Blah" >>$EMAILMESSAGE
# send an email using /bin/mail
/bin/mail -s "$SUBJECT" "$EMAIL" < $EMAILMESSAGE