UrFix's Blog – A geek without a cause

The pickaxe

The Pickaxe

tear down this house. A hundred thousand new houses
can be built from the transparent yellow carnelian

buried beneath it, and the only way to get to that
is to do the work of demolishing and then

digging under the foundations. With that value
in hand all the new construction will be done

without effort. And anyway, sooner or later this house
will fall on its own. The jewel treasure will be

uncovered, but it won’t be yours then. The buried
wealth is your pay for doing the demolition,

the pick and shovel work. If you wait and just
let it happen, you’d bite your hand and say,

“I didn’t do as I knew I should have.” This
is a rented house. You don’t own the deed.

You have a lease, and you’ve set up a little shop,
where you barely make a living sewing patches

on torn clothing. Yet only a few feet underneath
are two veins, pure red and bright gold carnelian.

Quick! Take the pickaxe and pry the foundation.
You’ve got to quit this seamstress work.

What does the patch-sewing mean, you ask. Eating
and drinking. The heavy cloak of the body

is always getting torn. You patch it with food,
and other restless ego-satisfactions. Rip up

one board from the shop floor and look into
the basement. You’ll see two glints in the dirt.

Rumi

5 Linux Commands: timeout, cpulimit,awk,tar and youtube-dl

I know how much you love random linux commands so here I’ve compiled some cool random linux commands to copy, convert, limit,kill and redirect things.

Start COMMAND, and kill it if still running after 5 seconds

timeout 5s COMMAND

Convert Youtube videos to MP3

youtube-dl -t --extract-audio --audio-format mp3 YOUTUBE_URL_HERE
youtube-dl has this functionality built in. If you’re running an older version of youtube-dl, you can update it using `youtube-dl -U` (although if you have an older version, it probably doesn’t download youtube videos anyway.)

youtube-dl –help will show you other options that may come in useful.

Limit the cpu usage of a process

sudo cpulimit -p pid -l 50
This will limit the average amount of CPU it consumes.

Target a specific column for pattern substitution

awk '{gsub("foo","bar",$5)}1' file
Awk replaces every instance of foo with bar in the 5th column only.

Redirect tar extract to another directory

tar xfz filename.tar.gz -C PathToDirectory
The command extracting the tar contents into particular directory …

Linux Command Line and MySQL: Awesome

MySQL is the world’s most popular open source database. Whether you are a fast growing web property, technology ISV or large enterprise, MySQL can cost-effectively help you deliver high performance, scalable database applications. Check out this site MySQL Commands for a nice MySQL cheat sheet.
UrFix.com however has created a list of commands I use almost daily when monitoring and maintaining my LAMP server. I hope you find these useful…

Monitor the queries being run by MySQL

watch -n 1 mysqladmin --user= --password= processlist

Watch is a very useful command for periodically running another command – in this using mysqladmin to display the processlist. This is useful for monitoring which queries are causing your server to clog up.
More info here: http://codeinthehole.com/archives/2-Monitoring-MySQL-processes.html

Backup all MySQL Databases to individual files

for I in $(mysql -e 'show databases' -s --skip-column-names);
do mysqldump $I | gzip > "$I.sql.gz"; done

I put this in a cron job to run @ midnight – “lazy back up”

Copy a MySQL Database to a new Server via SSH with one command

mysqldump --add-drop-table --extended-insert --force --log-error=error.log
 -uUSER -pPASS OLD_DB_NAME | ssh -C user@newhost "mysql -uUSER -pPASS NEW_DB_NAME"

Dumps a MySQL database over a compressed SSH tunnel and uses it as input to mysql – i think that is the fastest and best way to migrate a DB to a new server!

Convert all MySQL tables and fields to UTF8

mysql --database=dbname -B -N -e "SHOW TABLES"  | awk '{print "ALTER TABLE", $1,
 "CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;"}' | mysql --database=dbname
&

Backup a remote database to your local filesystem

ssh user@host 'mysqldump dbname | gzip' >
/path/to/backups/db-backup-`date +%Y-%m-%d`.sql.gz

I have this on a daily cronjob to backup the urfix.com database from NearlyFreeSpeech.net (awesome hosts by the way) to my local drive. Note that (on my Ubuntu system at least) you need to escape the % signs on the crontab.

Export MySQL query as .csv file

echo "SELECT * FROM table; " | mysql -u root -p${MYSQLROOTPW}
databasename | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > outfile.csv

This command converts a MySQL query directly into a .csv (Comma Seperated Value)-file.

Create an SSH tunnel for accessing your remote MySQL database with a local port

ssh -CNL 3306:localhost:3306 user@urfix.com

Count the number of queries to a MySQL server

echo "SHOW PROCESSLIST\G" | mysql -u root -p | grep "Info:" |
awk -F":" '{count[$NF]++}END{for(i in count){printf("%d: %s\n",
count[i], i)}}' | sort -n

dump a single table of a database to file

mysqldump -u UNAME -p DBNAME TABLENAME> FILENAME

And there you have it, a nice list of hopefully useful commands that you can inspect and learn from.

8 Cool Ways To Use SCP

The SCP protocol is a network protocol, based on the BSD RCP protocol, which supports file transfers between hosts on a network. SCP uses Secure Shell (SSH) for data transfer and utilizes the same mechanisms for authentication, thereby ensuring the authenticity and confidentiality of the data in transit. A client can send (upload) files to a server, optionally including their basic attributes (permissions, timestamps). Clients can also request files or directories from a server (download). SCP runs over TCP port 22 by default. Like RCP, there is no RFC that defines the specifics of the protocol.

SCP is an awesome tool. Learn it, Love it, Use it….

Edit a file on a remote host using vim

vim scp://username@host//path/to/somefile

Colored diff ( via vim ) on 2 remotes files on your local computer.

vimdiff scp://root@server-foo.com//etc/snmp/snmpd.conf scp://root@server-bar.com//etc/snmp/snmpd.conf

Restrict the bandwidth for the SCP command

scp -l10 user@urfix.com:/home/urfix/* .

the command is obvious, I know, but maybe not everyone knows that using the parameter “-l” you can limit the use of bandwidth command scp.
In this example fetch all files from the directory zutaniddu and I copy them locally using only 10 Kbs

Compare a remote file with a local file

vimdiff  scp://[@]/

Easily scp a file back to the host you’re connecting from

mecp () { scp "$@" ${SSH_CLIENT%% *}:Desktop/; }

Place in .bashrc and invoke like this: “mecp /path/to/file”, and it will copy the specified file(s) back to the desktop of the host you’re ssh’ing in from. To easily upload a file from the host you’re ssh’ing in from use this:

ucp (){ scp ${SSH_CLIENT%% *}:Desktop/upload/* .; }

scp file from hostb to hostc while logged into hosta

scp user@hostb:file user@hostc:

While at the command line of of hosta, scp a file from remote hostb to remote hostc. This saves the step of logging into hostb and then issuing the scp command to hostc.

Copy something to multiple SSH hosts with a Bash loop

for h in host1 host2 host3 host4 ; { scp file user@$h:/destination_path/ ; }

Just a quick and simple one to demonstrate Bash For loop. Copies ‘file’ to multiple ssh hosts.

scp with compression.

scp -C 10.0.0.4:/tmp/backup.sql /path/to/backup.sql

-C is for compression.