Archive for category Technology
Reducing IT failures with Open Source
Posted by Administrator in Technology on December 31st, 2009
A recent ZDNet article describes the impact IT project failures on our economy… and the numbers are scary. The author estimates that $6.2 trillion dollars is lost annually worldwide. What does that really mean? While ultimiately this number a SWAG (sophisticated wide-ass guess), it does illustrating the high rate of failures within the IT field.
I believe these failures can we reduced though better utilization of existing technology, specifically though the magic of open source. Open source software is developed with the intention of being sharing with the general public. This results in code which has been peer reviewed resulting in an arguably more secure final product.
A problem I frequently encounter are managers who want to “reinvent the wheel”. The result is less time is spent on quality control, detailed planning, and true development. The Internet is teeming with thousands of open source projects covering all disciplines, which most developers already know. These projects can be used to fill in gaps, and cut out large chucks of development time. As an example, there are at least a half dozen legitimate content management system which are completely open source. When commissioned to build a website, does it really make sense to build a custom CMS? Doubtful.
So why does the wheel get redesigned every day? Generic managers. Typically managers do not fully understand how to utilize open source and by their nature like to keep everything under their control.
The solution? Train developers to be managers. The same way a factory worker trains to become a line manager — IT developers must establish a place in management. As technology continues to grow in sophistication, the way we develop needs to evolve and mature. Eliminate managers who cannot develop and you will significantly reduce IT failure rates. I guarantee it.
Dell Laptop XPS M1710 SUCKS!
Posted by Administrator in Technology on July 15th, 2009
Here we go again, another crappy Dell laptop with serious design issues. This heat generating monster should never have been stuffed into a laptop form factor. After multiple fan failures the video card is now toast and the laptop will no longer display anything. The best part? It costs more to buy a replacement video card then what the machine is worth.
Message to Dell – Quality matter, just because you CAN build something cheap doesn’t mean you should. Laptops shouldn’t burn your lap and definately don’t need tacky glowing LED lights (producing more heat and wasting power).
Mac Mini? The coolest computer ever!
Posted by Administrator in Hardware, Technology on July 15th, 2009
Mac Mini? The coolest computer ever!
Apple has always been known for outstanding quality, however they’re use has been limited to a select group of dedicated “Mac People”. The landscape is changing… PC’s are no longer limited to beige box towers. The introduction of the iPhone shows how computers are jumping into different arenas. The Mac Mini is an illustration of how our perception of a computer is all wrong and we need to re-evaluate our assumptions and expectations. Computers don’t have to be noisy and don’t have to waste space and power. The Mac Mini makes a prefect media center machine and can be used as a server as well. It’s no surprise that there are companies selling Mac Mini’s within data centers (Mac Mini Hosting) as a replacement for expensive rack mount servers. These little machines have a genius design! They use very little power and have no annoying loud fans. Apple is really on top of their game when it comes to envisioning where the market is headed.
Windows XP broken .exe and .lck file association
Posted by Administrator in Technology on May 5th, 2009
I recently repaired a Windows XP installation for a friend. The machine would not boot into Windows and was complaining about a missing system files. Running the Windows XP repair tool resolved the problem, however LCK and EXE files would still not work correctly.
The solution can be found at the following URL:
http://www.dougknox.com/xp/file_assoc.htm
This site also has registry fixes for a bunch of system file associations. In my case running the EXE and LCK repairs worked prefect.
Fighting off a DDOS attack on an Apache web server
Posted by Administrator in Hardware, Technology on April 29th, 2009
When it rains it pours! It’s hard enough to keep websites running without hackers trying to break in. An even worse situation is when hackers simply want you off-line. A few days ago I experienced just that… my first DDOS (distributed denial of service) attack against a high-profile website we host.
The attacker original exploited some legacy code to gain access to the web sites administration tool. After thwarting their attack, our web server immediately started to hit “max client connections”. Even after restarting Apache the max client connections were reached within seconds.
We quickly created bad routes for IP addresses associated with the hack attempts; however this did not resolve the problem. Below is the syntax we used to add bad routes.
Route add –host xxx.xxx.xxx.xxx reject
The Apache log files left no clues as to who was connecting so we had to look elsewhere. After some quick searching we found a terrific PHP script which shows which IP’s are connecting to your web server too much. The script was just want we needed. After about a ½ hour of blocking listed IP addresses the problem went away. In all we blocked a little over 100 IP addresses.
If you’re ever in a situation where Apache is clearly being flooded with connections. Run this script! It will save you a ton of time identifying the attacker(s).
– BEGIN SCRIPT –
<?php
## Functions ##
function getIP($line) {
ereg("[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}",$line,$regMatch);
$ip = $regMatch[0];
if($ip) return $ip; else return "false";
}
function processString($string, $size = 18) {
$string = "[ ".$string;
$length = strlen($string);
$toAdd = $size - $length;
for($x = 0; $x < $toAdd; $x++) {
$string = $string." ";
}
$string = $string."]";
return $string;
}
## Code ##
while (true) {
$cmd = "netstat -n | awk '{ print $5 }'";
exec($cmd, $netstatArray);
$ipArray = array();
foreach($netstatArray as $line) {
$ip = getIP($line);
if($ip != "false" && ip != "127.0.0.1") {
if(array_key_exists($ip, $ipArray))
{
$ipArray[$ip]+=1;
}
else // if not, count=1
{
$ipArray[$ip] = 1;
}
}
}
asort($ipArray);
system("clear");
foreach($ipArray as $ip => $count) {
if ($count < 15)
continue;
echo processString($ip);
echo "\t" .processString(gethostbyaddr($ip), 55);
echo "\tTimes Accessed: " .$count ."\n";
}
echo str_repeat("-", 50) ."\n";
exec("top -n 1", $top_str);
preg_match("#load average:(.+)#i", $top_str[0], $match);
echo "Load Average: " .$match[1] ."\n";
echo str_repeat("-", 50) ."\n";
echo 'Showing $count >= 15: (Escape with ctrl+c)' ."\n";
sleep(10);
}
?>
Using gddrescue to save your data – Part 2
Posted by Administrator in Technology on April 10th, 2009
After 2 weeks of running gddrescue I was able to recover all important files from the disk. Depending on how much data is missing, photorec is probably the best tool to use. If the disk was too damaged to boot its likely not going to be re-mountable. photorec scans the raw image looking for specific file signatures. This process is referred to as “file craving”. Originally developed to recover images from damaged flash media, photorec works prefect for finding Microsoft Office files on a recovered image.
I was also successful in merging two images together. gddrescue allows you to define the start position and attempts to rebuild the whole image. If you start at block 10,000,000 the image will leave “white” space for the first 9,999,999 blocks. This allows you to combine multiple chunks together. In my situation I had an image created with dd_rescue which needs applied to a more complete gddrescue image. All I had to do was point to image and define the start / end position.
Using gddrescue to save your data – Part 1
Posted by Administrator in Technology on March 18th, 2009
My experience with ddrescue and dd_rescue
About a week ago my wife’s laptop hard drive failed sudden. She had not been making regular backups of her data so now the goal was to restore as much data as possible as quickly as possible.
My first attempts at recovering the data were weak. My gut instinct was to reach for my trusty hirum boot CD which has a variety of recovery tools. Sadly, most of these tools either didn’t work or locked up when trying to read from the disk. It was time to get more serious…
After doing some additional research I found dd_rescue, a bulk copy tool similar to UNIX / Linux “dd”, but is designed to recover gracefully from disk errors. I was thrilled to find such a simple application specifically designed for this problem. I left the disk connected to a machine to recover data over night.
The next morning I found the entire process failed around 5GB. I tried to cancel and restart the process, but apparently dd_rescue doesn’t handle interruption very well. After some investigation I found a similar application called gddrescue. This package comes directly from the GNU foundation and has the added feature of defining a “log” so you can easily pickup where you left off. Clearly gddrescue has more functionality and is superior to dd_rescue.
I started the recovery around the point of failure using gddrescue now. This time when the process was interupted starting back up was easy. After about 7 days of work the drive is almost finished, however its missing the first 10,000,000 blocks from my dd_rescue image. It should be possible to merge the devices together, I’ll discuss that in part 2…
Here is a quick summary of how to use gddrescue /w Knoppix boot CD.
- Burn a copy of the latest Knoppix boot CD.
- Install your damaged disk in another computer, make sure the BIOS sees both disks.
- Boot using Knoppix and mount your good hard drive. Make sure you have more space then the entire size of the disk your trying to recover.
- Here is where it gets tricky, Knoppix doesn’t come with gddrescue, only dd_rescue. You can install gddrescue using a *.deb package.
-
Execute the following command ' ddrescue /dev/hdb1 (damaged disk) /media/disk1/recovery.img (good disk) /media/disk1/log_file '
- Watch and wait. If you have to cancel simply re-execute the command.
A few warnings about data recovery…
- Whatever you do do not write any more data to a damaged disk.
- Work quickly to extract an image of the disk. Run recovery tools against the image, not the damaged disk.
- Make backups so you don’t have to go through this.
Part 2 will cover the following…
- The use of file “carving” software to extract data from a raw partition.
- Merging two images together using gddrescue.
- The outcome of my week long recovery effort.
Grandstream HandyTone 286 Review
Posted by dwright in Communications, Hardware, Technology on February 27th, 2009
After my Digium AIXy 101 suddenly died I was left to find another ATA (Analog Telephone Adapter) for my home PBX. After doing some research I settled on the cheapest solution, the Grandstream HandyTone 286. Its a terrific little device which required very little effort to setup.
The web interface leaves a lot to be desired, but giving the price-point I believe its an outstanding value. The device was easy to locate in my routers DHCP and ARP tables. After upgrading I’ve found the calls sound better and connect faster.
The only pitfall is the device only uses SIP rather then my preferred method of IAX2. IAX2 has the benefit of transversing NAT / PAT without much trouble. SIP on the other hand is a major pain. Luckily the SIP server was on the same LAN so this was not an issue for me. The device does support STUN to allow proxying of SIP connections if you need to traverse NAT.
I would recommend this device to anyone looking to connect analog phones to a PBX using SIP. I have not tried the rest of their line of HandyTone products but look forward to giving them a chance.
AWstats in a clustered environment
Posted by dwright in Software, Technology on February 17th, 2009
Below is a simple solution for operating AWstats within a clustered environment in CentOS 5. This solution should work for other operating systems as well. This explanation assumes you’ll be using two application nodes, with front-end directors. I have not tried this solution using three nodes.
- Install AWstats & NFS from yum repository.
- Create profile for your domain within /etc/awstats. (i.e. awstats.www.domain.com.conf)
- Configure NFS mounts on each node. This can be accomplished by modifying your /etc/fstab file as illustrated below.
- Adjust your awstats configuration file to point at both log sources now that the NFS mount has been created.
- On second node, mount /var/lib/awstats
- Configure nightly cron on first application node.
- Configure calling either awstats.pl should display the data processed data stored on node 1, regardless of which node the user is accessing.
Notes / Exceptions
Be sure to backup your Apache log files, and your AWstats data. In this solution the processed information will only be stored on the primary node
Full Setup Process
1. SSH APP1
2. yum install awstats nfs -y
3. nano /etc/exports
4. Add following lines:
/var/log/httpd APP2(rw,sync)
/var/lib/awstats APP2(rw,sync)
/etc/awstats APP2(rw,sync)
5. nano /etc/fstab
6a. Add lines, APP2:/var/log/httpd /var/log/httpd-APP2 nfs rsize=16384,wsize=16384 0 0
6b. mount /var/log/httpd-APP2
7. ln -s /var/log/httpd /var/log/httpd-APP1
8. service nfs start
9. chkconfig nfs on
10. cp /etc/awstats/awstats.model.conf /etc/awstats/awstats.www.domain.com.conf
11. nano /etc/awstats/awstats.www.domain.com.conf
12. Configure normally, exception add
LogFile="/usr/local/awstats/tools/logresolvemerge.pl /var/log/httpd-APP1/domain.com-access_log \
/var/log/httpd-APP2/domain.com-access_log |"
13. SSH APP2
14. yum install awstats nfs -y
15. nano /etc/exports
16. Add following lines:
/var/log/httpd APP1(rw,sync)
/var/lib/awstats APP1(rw,sync)
/etc/awstats APP1(rw,sync)
17. nano /etc/fstab
18a. Add following lines,
APP1:/var/log/httpd /var/log/httpd-APP1 nfs rsize=16384,wsize=16384 0 0
APP1:/var/lib/awstats /var/lib/awstats nfs rsize=16384,wsize=16384 0 0
18b. mount /var/log/httpd-APP1
18c. mount /var/lib/awstats
19. ln -s /var/log/httpd /var/log/httpd-APP2
20. service nfs start
21. cp /etc/awstats/awstats.model.conf /etc/awstats/awstats.www.domain.com.conf
22. nano /etc/awstats/awstats.www.domain.com.conf
23. Configure normally, exception add
LogFile="/usr/local/awstats/tools/logresolvemerge.pl /var/log/httpd-APP1/domain.com-access_log \
/var/log/httpd-APP2/domain.com-access_log |"
24. SSH APP1
25. crontab -e add line, perl /usr/local/awstats/tools/awstats_updateall.pl now
26. Configure Apache to expose awstats.pl, refer to awstats setup for additional information.
27. Confirm setup (df -h, perl awstats.pl -update -config=www.domain.com)
This should allow users accessing either node to display the same stats information compiled from both nodes access logs. Be sure to do backups!
Enjoy,
Dustin
Routing public IP addresses across via virtual tunnel
Posted by dwright in Communications, Software, Technology on May 2nd, 2007
A recent situation came up which required that I move a group of servers off-site without changing their public IP address. The off-site location is behind my cable modem at my house. To solve the problem I enlisted the help of an open source application called vtun or Virtual Tunnel and IP tables on a standard CentOS 4.4 installation. Below is a outline of the steps I took to solve the problem.
-> Build two CentOS 4.4 Linux routers. Minimal installation plus development tools. In my situation I used a dual 350Mhz PII and tunnel router #2 was an old Pentium Pro 200Mhz. The dual 350Mhz is located at the main data center and the older Linux box is located at my house.
-> Download and configure vtund from www.vtund.info. There are a number of example configurations available on their website. Below is what I have used.
# IP Tunnel Server Configuration
cobra {
passwd XXXXXXX; # Password
type tun; # IP tunnel
proto udp; # UDP protocol
compress lzo:9; # LZO compression level 9
encrypt no; # Encryption
keepalive yes; # Keep connection alive
up {
# Connection is Up
# XXX.XXX.XXX.XXX – local, XXX.XXX.XXX.XXX – remote
ifconfig “%% XXX.XXX.XXX.XXX pointopoint XXX.XXX.XXX.XXX mtu 1450″;
};
down {
# Connection is Down
# Shutdown interface
ifconfig “%% down”;
};
}
# IP Tunnel Client Configuration
cobra {
passwd XXXXXXX; # Password
device tun0; # Device tun0
persist yes; # Persist mode
up {
# Connection is Up
# Assign IP addresses.
ifconfig “%% XXX.XXX.XXX.XXX pointopoint XXX.XXX.XXX.XXX mtu 1450″;
};
}
The configuration above is very simlar to the example configurations provided by vtund. There is more you can do here to make this connection more robust but for the purpose of this blog entry I’ll show it works in more detail. To make the connection between to the two servers you run the following command on the client machine.
vtund cobra
This should create a new network interface called tun0 on both servers. This will allow you to connect across the tunnel. To test this i recommend using the following commands.
On the client
tcpdump -i tun0
On the server
ping
You should see packets come across the interface and reply back. Once you have established the tunnel connection there is a bit more required to route public IP address both in and out of the tunnel interface. For the sake of this example we’ll use 10.10.10.0/24 as the network block your routing. You’ll want to be using a public IP block for this to work correctly not the interal block from the example!
-> Assuming 10.10.10.0/24 is already routed to the tunnel server you’ll now need to route to packets from the tunnel server to the client. To do this you simply type:
ip route add 10.10.10.0/24 dev tun0
This will push packets from the server -> client. Once the packets are on the client end of the tunnel however they do not have a correct route back to the sender. In my situation the tunnel client is located on a cable modem with a default gateway on a different network. My block of 10.10.10.0/24 cannot be routed across someone elses network. The solution is to flag packets as they enter into the tunnel client router. Once the packets get flagged we can apply an alternate route table to them. The alternate route cable is identical to the standard table except the default gateway is tun0 rather then the cable modem interface. Below is a simple script to generate a duplicate route table and add the make the default gateway the tunnel server:
ip route show table main | grep -Ev ^default | while read ROUTE ; do
ip route add table 200 $ROUTE
done
ip route add default via XXX.XXX.XXX.XXX table 200
Now that the alternate table has been created you can verify it works by typing:
ip route list table 200
This should be a simlar table to your main route table except the default gateway should be the IP address of the server-side of the tunnel interface. Now that this table is in place we need to start flagging packets. In order to do this we need to make some special rules in ip tables. Below are the commands to create the flagging rules:
iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark=200
iptables -t mangle -A PREROUTING -s 10.10.10.0/24 -j MARK --set-mark=200
iptables -t mangle -A OUTPUT -s 10.10.10.0/24 -j MARK --set-mark=200
Once the iptable rules are in place you can check them by issuing the ‘iptables -nvL -t mangle’ command. In the output you should see packets hitting your new iptable rules. Once you have active hits you’ll need to add a rule that all packets with a flag need to use the alternate table. To create this rule you need the following
ip rule add fwmark 200 table 200
To verify the rule was added you can issue the ‘ip rule list’ command.
Once you have does this your ready to route packets in and out of the tunnel. In my situation I added a third network interface card running NAT for my home computers.