Skip navigation

Okay, Kris kept drumming this one into me.  It was late, I was tired, but I did remember it and I managed to reclaim perhaps 20 mins today using it and perhaps hours moving forward.

Picture this.  You’re working with the terminal on a remote box.  You need to change n number of occurrences of one string with another.  You don’t have a pretty GUI.

Your choices:

1, You download the file to your local box, make the changes and then upload it.  Perhaps you GUI it all the way.  GUI FTP  > GUI Editor > GUI FTP

2, The hardcore geek way.  Using VIM, you apply the following command  :%s/meadowbrook/www.meadowbook-cottage.co.uk/g

Hey, we’re geeks.  we’ve got to do it the hardcore geek way!

Here, meadowbrook is the “needle”, the search string. “www.meadowbrook-cottage.co.uk” is the replacement string. The “/g” is greedy.  It replaces every occurrence.

So, thanks, Kris.  You helped me claw back time. :)

 

 

I’m not a sysadmin or networks guy, obviously.  But here are some tools I’ve only touched on, but found invaluable while troubleshooting DNS and Firewall related problems.

ping example.com is useful to quickly determine whether or not a domain can be resolved to an IP.  (Provided it is supported server-side, it doesn’t use ports, but ICMP message)

If the domain doesn’t resolve and you have a static IP address for the box, ping that to confirm the box is up and running.

If it is, you’ve got a DNS issue.

dig example.com will provide lots of information, like the nameservers etc.

nmap example.com will perform a network scan and report back which ports are open.  Here is an example on one of my domains:

nmap blessmycottonsocks.co.uk

Starting Nmap 5.21 ( http://nmap.org ) at 2012-03-20 17:08 GMT
Nmap scan report for blessmycottonsocks.co.uk (92.48.68.148)
Host is up (0.051s latency).
Not shown: 997 filtered ports
PORT    STATE SERVICE
22/tcp  open  ssh
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 9.70 seconds

traceroute is interesting because it will illustrate how many hops your request goes through in order to get to the destination server:

traceroute blessmycottonsocks.co.uk
traceroute to blessmycottonsocks.co.uk (92.48.68.148), 30 hops max, 60 byte packets
1  192.168.0.1 (192.168.0.1)  0.754 ms  2.426 ms  2.513 ms
2  88-111-0-1.dynamic.dsl.as9105.com (88.111.0.1)  36.480 ms  39.433 ms  42.813 ms
3  xe-4-0-0.ner001.the.as13285.net (80.40.155.102)  46.501 ms  49.814 ms  52.647 ms
4  80.40.155.107 (80.40.155.107)  55.821 ms 80.40.155.111 (80.40.155.111)  58.195 ms 80.40.155.107 (80.40.155.107)  60.834 ms
5  xe-10-3-0.bragg002.log.as13285.net (80.40.155.39)  63.798 ms xe-9-3-0.bragg001.log.as13285.net (80.40.155.37)  66.548 ms xe-10-3-0.bragg001.log.as13285.net (80.40.155.35)  73.250 ms
6  xe-7-0-0.scr001.loh.as13285.net (80.40.155.62)  74.473 ms xe-7-3-0.scr001.log.as13285.net (80.40.155.52)  45.179 ms xe-5-0-0.scr001.loh.as13285.net (80.40.155.60)  38.862 ms
7  host-78-144-0-125.as13285.net (78.144.0.125)  70.552 ms host-78-144-0-137.as13285.net (78.144.0.137)  43.954 ms  46.290 ms
8  hex89-linx.as29550.net (195.66.236.223)  51.540 ms the-linx.as29550.net (195.66.224.223)  52.975 ms hex89-linx.as29550.net (195.66.236.223)  55.371 ms
9  a.6.magic-hex.as29550.net (213.229.122.210)  61.411 ms  64.121 ms vl668.hex-7600.as29550.net (92.48.95.13)  64.188 ms
10  * a.6.magic-hex.as29550.net (213.229.122.210)  76.266 ms *
11  * * *
12  * * *

I lost 4 hours of my life today trying to work out why VirtualHosts on Apache wasn’t resolving to the correct domains on a new box.  It wasn’t helped by a poorly configured BIND DNS server on the same box or the fact that the smallest changes take time to propogate across the web.

I tried one thing after another after another.  Scratched my head.  Tried different things again and again and again.

In the end, I resolved it.  Was it the DNS server I ripped out after 3 hours of despair?  No.  Had I installed anything incorrectly? No.  Was there some mysterious “on” switch I forget to turn?  No.

I simply forgot to add “www” to the ServerAlias for each of the domains.  Well, I say, forgot.  I didn’t realise I needed to add it.  Why?  Because I’d used files from my local development server as a guide.

<VirtualHost *:80>
ServerName lazylover.co.uk
DocumentRoot /var/www/lazylover.co.uk
ServerAlias www.lazylover.co.uk

<Directory /var/www/lazylover.co.uk/>
Options Indexes FollowSymLinks Multiviews
AllowOverride All
Order allow,deny
allow from all
</Directory>
</VirtualHost>

This was an expensive mistake.

 

 

Iptables comes shipped with Ubuntu, but without any rules.

Basically, you can control the flow of bytes passing through ports.  It’s structured around INPUT (data flowing into the box), OUTPUT (data flowing from the box somewhere else) and FORWARD (instead of dealing with the data, divert or forward it somewhere else).

You say what can come in and where, or not, as the case may be.  For example, TCP on port 22.

Useful commands:

iptables -L -v

This will give you verbose output about your existing rules.

iptables -h

This will explain how to use it.

Useful flags

-L lists all the rules

-A append a rule

-I insert a rule (useful as you add additional functionality.  Pass in the numeric position too)

 

It’s important to remember that the rules added will be forgotten on reboot.  The approach I’ve been advised to take is to create a shell script that loads the rules on reboot.  It’s also a good idea to write a script to write the latest changes too.

Here is what I did.

First, save the Firewall rules:

sudo sh -c "iptables-save > /etc/iptables.rules"

Then create a shell script to load them on reboot (/etc/network/if-pre-up.d/iptablesload)
#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0

don't forget to chmod +x /etc/network/if-pre-up.d/iptablesload

Then write the shell script to save any changes (/etc/network/if-post-down.d/iptablessave):

#!/bin/sh
iptables-save -c > /etc/iptables.rules
if [ -f /etc/iptables.downrules ]; then
   iptables-restore < /etc/iptables.downrules
fi
exit 0

Here is a useful tutorial from the guys at Ubuntu: https://help.ubuntu.com/community/IptablesHowTo.  In this tutorial they also give example rules.

Another piece of useful advice I got was to ensure that you always have a couple of terminals open when changing iptables rules.  For example, you don’t want to find that you can’t get back in after applying the wrong rule.

We all know it’s important to have indexes on our tables in order to speed up things.

Often, we take care of this transparently when we use a GUI tool like MySQL Workbench.

One of my colleagues, however, thinks we should only ever apply an index once the table has been up and running for a while so we can actually see how it’s being used in the field.

With this approach in mind, let’s just remind ourselves how to add an index the good old fashioned way.

CREATE INDEX some_index ON some_table(some_table_field);

We might also want to drop an unused index:

DROP INDEX some_index ON some_table;

Using PHPMyAdmin and/or Mysql Workbench, it’s easy to forget the basics:

CREATE USER 'someUser'@'localhost' INDENTIFIED BY 'somePassword';

GRANT ALL  ON  *.*  TO 'someUser'@'localhost';

If you want to restrict what ‘someUser’ can do to say, reading and writing to a particular database:

GRANT SELECT, INSERT, DELETE, UPDATE ON 'someDatabase'.* TO 'someUser';

Very occasionally, you might be searching for something where you need to use two or more patterns – where each pattern is applied one after another.  This might be because if you used the last pattern only, it would return too many results or incorrect results.

In this example, imagine movies.txt contains thousands of movies and I want to know how many movies there are with Akira in the title and of those matches, how many use the video format avi.

grep Akira movies.txt | grep -c .avi

I once had a need to count the number of times a new line started with 01 in a text file.  I can’t remember why I needed this, but I must have had a good reason for it at the time. However, I do remember that I ended up using grep because the file was really, really big and when I tried to open it with a conventional editor it caused my desktop PC to crash.

grep -c '^01' path/to/file

 

Okay, a very quick and dirty way of finding a string of text in one or more files recursively is to use grep.

I regularly do this if I’m trying to find a variable in one or more other PHP scripts when my Eclipse IDE isn’t quite up to the job.

grep -R someString .

AB  is the Apache HTTP server benchmarking tool.

In order to install on Linux/Ubuntu:

sudo apt-get install apache2-utils

Here is an example of bench marking a website:

ab -c 2 -n 100 http://path/to/your/site

Here, -c 2 refers to 2 simultaneous connections and -n 100 refers to the number of times.