5 Linux Commands: timeout, cpulimit,awk,tar and youtube-dl

I know how much you love random linux commands so here I’ve compiled some cool random linux commands to copy, convert, limit,kill and redirect things.

Start COMMAND, and kill it if still running after 5 seconds

timeout 5s COMMAND

Convert Youtube videos to MP3

youtube-dl -t --extract-audio --audio-format mp3 YOUTUBE_URL_HERE
youtube-dl has this functionality built in. If you’re running an older version of youtube-dl, you can update it using `youtube-dl -U` (although if you have an older version, it probably doesn’t download youtube videos anyway.)

youtube-dl –help will show you other options that may come in useful.

Limit the cpu usage of a process

sudo cpulimit -p pid -l 50
This will limit the average amount of CPU it consumes.

Target a specific column for pattern substitution

awk '{gsub("foo","bar",$5)}1' file
Awk replaces every instance of foo with bar in the 5th column only.

Redirect tar extract to another directory

tar xfz filename.tar.gz -C PathToDirectory
The command extracting the tar contents into particular directory …

Linux Command Line and MySQL: Awesome

MySQL is the world’s most popular open source database. Whether you are a fast growing web property, technology ISV or large enterprise, MySQL can cost-effectively help you deliver high performance, scalable database applications. Check out this site MySQL Commands for a nice MySQL cheat sheet.
UrFix.com  however has created a list of commands I use almost daily when monitoring and maintaining my LAMP server. I hope you find these useful…

Monitor the queries being run by MySQL

 

watch -n 1 mysqladmin --user= --password= processlist

Watch is a very useful command for periodically running another command – in this using mysqladmin to display the processlist. This is useful for monitoring which queries are causing your server to clog up.
More info here: http://codeinthehole.com/archives/2-Monitoring-MySQL-processes.html

Backup all MySQL Databases to individual files

 

for I in $(mysql -e 'show databases' -s --skip-column-names);
do mysqldump $I | gzip > "$I.sql.gz"; done

I put this in a cron job to run @ midnight – “lazy back up”

Copy a MySQL Database to a new Server via SSH with one command

 

mysqldump --add-drop-table --extended-insert --force --log-error=error.log
 -uUSER -pPASS OLD_DB_NAME | ssh -C user@newhost "mysql -uUSER -pPASS NEW_DB_NAME"

Dumps a MySQL database over a compressed SSH tunnel and uses it as input to mysql – i think that is the fastest and best way to migrate a DB to a new server!

Convert all MySQL tables and fields to UTF8

 

mysql --database=dbname -B -N -e "SHOW TABLES"  | awk '{print "ALTER TABLE", $1,
 "CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;"}' | mysql --database=dbname
&

Backup a remote database to your local filesystem

 

ssh user@host 'mysqldump dbname | gzip' >
/path/to/backups/db-backup-`date +%Y-%m-%d`.sql.gz

I have this on a daily cronjob to backup the urfix.com database from NearlyFreeSpeech.net (awesome hosts by the way) to my local drive. Note that (on my Ubuntu system at least) you need to escape the % signs on the crontab.

Export MySQL query as .csv file

 

echo "SELECT * FROM table; " | mysql -u root -p${MYSQLROOTPW}
databasename | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > outfile.csv

This command converts a MySQL query directly into a .csv (Comma Seperated Value)-file.

Create an SSH tunnel for accessing your remote MySQL database with a local port

 

ssh -CNL 3306:localhost:3306 user@urfix.com

Count the number of queries to a MySQL server

 

echo "SHOW PROCESSLIST\G" | mysql -u root -p | grep "Info:" |
awk -F":" '{count[$NF]++}END{for(i in count){printf("%d: %s\n",
count[i], i)}}' | sort -n

dump a single table of a database to file

 

mysqldump -u UNAME -p DBNAME TABLENAME> FILENAME

And there you have it, a nice list of hopefully useful commands that you can inspect and learn from.

8 Cool Ways To Use SCP

The SCP protocol is a network protocol, based on the BSD RCP protocol, which supports file transfers between hosts on a network. SCP uses Secure Shell (SSH) for data transfer and utilizes the same mechanisms for authentication, thereby ensuring the authenticity and confidentiality of the data in transit. A client can send (upload) files to a server, optionally including their basic attributes (permissions, timestamps). Clients can also request files or directories from a server (download). SCP runs over TCP port 22 by default. Like RCP, there is no RFC that defines the specifics of the protocol.

SCP is an awesome tool. Learn it, Love it, Use it….

Edit a file on a remote host using vim

vim scp://username@host//path/to/somefile

Colored diff ( via vim ) on 2 remotes files on your local computer.

vimdiff scp://root@server-foo.com//etc/snmp/snmpd.conf scp://root@server-bar.com//etc/snmp/snmpd.conf

Restrict the bandwidth for the SCP command

scp -l10 user@urfix.com:/home/urfix/* .

the command is obvious, I know, but maybe not everyone knows that using the parameter “-l” you can limit the use of bandwidth command scp.
In this example fetch all files from the directory zutaniddu and I copy them locally using only 10 Kbs

Compare a remote file with a local file

vimdiff  scp://[@]/

Easily scp a file back to the host you’re connecting from

mecp () { scp "$@" ${SSH_CLIENT%% *}:Desktop/; }

Place in .bashrc and invoke like this: “mecp /path/to/file”, and it will copy the specified file(s) back to the desktop of the host you’re ssh’ing in from. To easily upload a file from the host you’re ssh’ing in from use this:

ucp (){ scp ${SSH_CLIENT%% *}:Desktop/upload/* .; }

scp file from hostb to hostc while logged into hosta

scp user@hostb:file user@hostc:

While at the command line of of hosta, scp a file from remote hostb to remote hostc. This saves the step of logging into hostb and then issuing the scp command to hostc.

Copy something to multiple SSH hosts with a Bash loop

for h in host1 host2 host3 host4 ; { scp file user@$h:/destination_path/ ; }

Just a quick and simple one to demonstrate Bash For loop. Copies ‘file’ to multiple ssh hosts.

scp with compression.

scp -C 10.0.0.4:/tmp/backup.sql /path/to/backup.sql

-C is for compression.

7 Awesome and Free Network Apps for Rooted Androids

Did you root your Android-based smartphone or tablet, thinking about it? Well I present you seven free network apps that take advantage of the superuser permissions. In the list I’ve composed , you’ll find apps that help with the network configuration and security of your device, apps to do sharing and sniffing on networks, and apps that demonstrate network hacking and security risks.

DroidSheep by Andreas Koch

What is this about?
If you know Firesheep or Faceniff, you probably know what this is about – one-click session hijacking using your android smartphone or tablet computer.
DroidsheepYou can use DroidSheep to demonstrate to yourself or others of how easy it is to take-over other user’s unsecured web sessions. You just open the app, agree to their disclaimer, and tap Start. You’ll start seeing the list populate once users on the network login to unsecured sites, displaying their URLs and session IDs. Then you can tap on an entry to open the site in full or mobile view, save the cookies, or export the cookies via email.

 

 

 

 

Samba File Sharing by funkyFresh

Samba file sharing

Access your Android device over wifi as a Windows shared folder.* Samba filesharing server for Android.
* Your Android device becomes visible to other Windows (& SMB compatible) computers on your network.
* The ‘external’ storage in your Android device is accessible as a network shared folder.
* Copy files and folders to and from your Android device using drag and drop.
* Customizable username, password, Windows workgroup name, and device network (NETBIOS) name.

Requirements:
* Your Android device must already be rooted.
* Superuser 2.3.6.2 or later installed (available on Market).

Set DNS by Mytechie

set dns

This app lets you easily define the name servers used by your Wi-Fi and mobile network interfaces. You can speed up browsing, enable content filtering, or bypass DNS-based filters by using a third-party DNS service. Android lets you natively change the DNS server addresses for just the Wi-Fi interface and is only available if you define static IP information. However, Set DNS lets you change both interfaces and can work when using DHCP.

 

 

 

 

 

Shark for Root by Elviss Kuštans

shark for root Traffic sniffer, works on 3G and WiFi (works on FroYo tethered mode too).his network sniffer is based on tcpdump and is basically a simple version of the popular WireShark application. It captures packet information from the Wi-Fi or 3G interfaces and automatically saves them to a .pcap file on the SD card. You can input tcpdump parameters to customize the sniffing sessions. You can view the dumps on your Android if you install another app (Shark Reader) or transfer the .pcap file to another computer to open in an application such as WireShark.

 

 

SSH Tunnel by Max Lv

sshtunnel for android This app helps you surf the web privately and securely, for instance, to encrypt your traffic from local eavesdroppers at Wi-Fi hotspots or to bypass filters and geographic restrictions. It’s a SSH client that can proxy the Internet connections for the entire Android system or individual apps through a SSH server. Its DNS Proxy feature can also solve the DNS pollution problem in places such as China by using the pre-defined HOSTS information from the developer’s site to counteract any DNS tampering by the authorities or ISPs.

 

 

 

WiFi Key Recovery by alt236

Android lets you modify the settings for Wi-Fi networks you’ve saved, but doesn’t show you the existing passwords, similar to Windows XP. However, this simple app reveals the login credentials for Wi-Fi networks stored on your device, useful if you forget the password to a network. It also demonstrates one of the security risks if you’re Android gets lost or stolen; someone can potentially get the credentials to log onto your home or work network.

 

 

 

Wireless Tether by Müller, Lemons, Buxton

wireless tether for androidThis lets you share your Android’s Internet connection with other computers or devices equipped with Wi-Fi or Bluetooth. It’s great if your Android version or mobile service provider doesn’t support tethering. For most Android devices, this app’s Wi-Fi sharing creates an ad-hoc (peer-to-peer) network. Some devices support the regular infrastructure-mode, which appears to look like a real Wi-Fi router or access point to the other devices. For either mode, it takes care of handing out IP addresses to clients via DHCP.

How To Enable Facebook Timeline Today – And A Preview TOO

This morning Facebook announced Timeline, an awesome look at everything that has  happened in your Facebook lifespan. It’s like a story book of your online life.

Facebook is enabling Timeline early for open graph developers. As a reader of UrFix Blog you’ll learn how to get started today.

1. Log into Facebook

2. Enable developer mode, if you haven’t already. To do this, type “developer” into the Facebook search box, click the first result (it should be an app made by Facebook with a few hundred thousand users), and add the app.

3. Jump into the developer app (if Facebook doesn’t put you there automatically, it should be in your left-hand tool bar)

4. Create a new app (don’t worry — you wont actually be submitting this for anyone else to see/use). Give your shiny new app any display name and namespace you see fit. Read through and agree to the Platform Privacy agreement. This is the step you need to be verified for.

5. Ensure you’re in your new app’s main settings screen. You should see your app’s name near the top of the page

6. Look for the “Open Graph” header, and click the “Get Started using open graph” link.

Create a test action for your app, like “read” a “book”, or “eat” a “sandwich”
7. This should drop you into an action type configuration page. Change a few of the default settings (I changed the past tense of “read” to “redd” — again, only you can see this unless you try and submit your application to the public directory), and click through all three pages of settings

8. Wait 2-3 minutes

9. Go back to your Facebook homescreen. An invite to try Timeline should be waiting at the top of the page

Only developers and yourself will be able to see your new profile till September 29th.

 

here is what mine looks like

and even a map of places I’ve check in….

Have fun, and remember this is still in beta so you are bound to find a bug or two.

Firefox 6 Is Out With A Few Changes

http://www.mozilla.com/en-US/firefox/new/
Firefox 6 is officially out, however the final build for all the platforms are now available for download from the official FTP channels. Much like the previous release, the changelog for Firefox 6 is quite flimsy, and the new build doesn’t feature any major new user facing feature. This is of course the side-effect of following a rapid release cycle. While it makes it easier for Mozilla to stick to the schedule, it also makes version numbers insignificant and immaterial. GHacks reported yesterday that Mozilla is planning on hiding the version number from normal users by removing it from the ‘About’ box. Of course, that would be an incredibly lame and stupid way to tackle the issue. The sensible thing for Mozilla would be to label these releases as minor version updates, and have one or two scheduled major updates per year that will actually deliver new features. Anyways, there is no point in blaming Mozilla alone. Google is the one who started this madness with their Chrome release cycle.


Coming back to Firefox 6, the biggest piece of new feature is that the address bar now highlights the domain of the website you are currently browsing. The site identity blocker has also received a minor facelift to make it sleeker than before. There are also a few behind the scenes improvements such as the support for WebSockets, improved Scratchpad, a new Web Developer menu item, an improved Web Console, and reduced browser startup time when using Panaroma.

There are as many as 1,300 changes in Firefox 6. However, almost all of them are bug fixes. When it comes to delivering new features, Firefox 6 disappoints, once again. If you have used Firefox 5, or even Firefox 4, you already know what to expect. If you liked either of those two releases, you will like Firefox 6. If you didn’t, then Firefox 6 will not change your opinion.

source http://techie-buzz.com/browsers/firefox-6-review.html

16 Cool Ass Random Linux Stuff

$ echo “Hello world”
Hello world

Urfix has compiled a sweet ass list of random stuff you can do from the command line. Hope you enjoy.

Fry: Make up some feelings and tell her you have them.

1 Multiple variable assignments from command output in BASH

read day month year <<< $(date +'%d %m %y')

This version uses read instead of eval.

2 Generate an XKCD #936 style 4 word password

shuf -n4 /usr/share/dict/words | tr -d '\n'

4 random words are better than one obfuscated word

http://xkcd.com/936/

3 All IP connected to my host

netstat -lantp | grep ESTABLISHED |awk '{print $5}' | awk -F: '{print $1}' | sort -u

find all computer connected to my host through TCP connection.

4 Ask for a password, the passwd-style

read -s -p"Password: " USER_PASSWORD_VARIABLE; echo

You can ask repeatedly for a non-blank password using this function:
function read_password() {
while [ ! -n "$USER_PASSWORD" ]; do
read -s -p”Password: ” USER_PASSWORD
if [ ! -n "$USER_PASSWORD" ]; then
echo “ERROR: You must specify a valid password, please try again”
fi
echo
done
}
Also you can set a time out (in seconds) to write the password
read -t 10 -s -p”Password: ” USER_PASSWORD_VARIABLE
if [ ! $? -eq 0 ]; then
echo “Time out!”
fi

5 Check syntax for all PHP files in the current directory and all subdirectories

find . -name \*.php -exec php -l "{}" \;

6mkdir & cd into it as single command

mkdir /home/foo/doc/bar && cd $_

The biggest advantage of this over the functions is that it is portable.

7 Merge PDFs into single file

gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=output.pdf input1.pdf input2.pdf ...

8 Create a QR code image in MECARD format

qrencode -o myqr.png 'MECARD:N:Daprophet,Isaiah;TEL:8881234567;EMAIL:isaiah.daprophet@urfix.com;;'

Add the QR code image on your webpage, business card ., etc, so people can scan it and quick add to their Contact Address Book. Tested on iPhone with QRreader.

9 Chmod all directories (excluding files)

find public_html/ -type d -exec chmod 755 {} +

+ at the end means that many filenames will be passed to every chmod call, thus making it faster. And find own {} makes sure that it will work with spaces and other characters in filenames.

10 List only the directories

find . -maxdepth 1 -type d | sort

11 copy with progress bar – rsync

rsync -rv   --progress

-r for recursive (if you want to copy entire directories)
src for the source file (or wildcards)
dst for the destination
–progress to show a progress bar

12 Find all files of a type and copy them elsewhere while keeping intact their full directory structure using find and cpio

find . -iname "*.flac" | cpio -pdm /Volumes/Music/FLAC

.flac is the filetype.
/Volumes/Music/FLAC is the destination.

13 Share a screen session

screen -x 

14 Backup entire system through SSH

ssh -C USER@HOST tar -c --exclude /proc --exclude /sys / | tar -x

15 Find the package a command belongs to on debian-based distros

apt-file search iostat

16 View files opened by a program on startup and shutdown

sudo lsof -rc command >> /tmp/command.txt

Run this before you run a command in order to see what the command does as it starts.
The -c flag is useful here as the PID is unknown before startup.
All config files, libraries, logs, ports, etc used by the command as it starts up, (and shuts down) will be captured at 1s intervals and written to a file.
Useful for debugging etc.

Install Mozilla Firefox 5 Or You Will Be Murdered!

Mozilla Firefox 5 has been released earlier this week, only three months after rolling out Firefox 4 and a month after it released version 5 in beta.
Version 5 has “more than 1,000 enhancements,” which include the “Do Not Track” privacy feature and support for the CSS Animations standard, among other things.
In its rush to make the Web better, however, Mozilla is taking criticism for not making it especially clear to users that it would stop issuing vulnerability patches for Firefox 4.
That has given rise to concerns that users who delay updating for various reasons may not realize they’ll lack protection against the latest malware.
“Firefox 5 is the security update for Firefox 4, and we do not plan to release a Firefox 4.0.2,” Johnathan Nightingale, the Mozilla Foundation’s director of Firefox engineering, told TechNewsWorld.
Should Mozilla have more forcefully notified Firefox 4 users that they have to upgrade to version 5? Should it include automatic updates instead of just sending users a pop-up window reminding them to update their browsers?

The Paradox of Speed and Security
The release cycle for new versions of browsers has been drastically shortened as the players seek to trump each other’s products with newer and better ones.
That bumped-up product cycle has both advantages and disadvantages.
“Security is typically the first area to be sacrificed when developers are under increased pressure to get out new software releases,” Stach & Liu’s Brown pointed out.
“The industry will need to be vigilant in scrutinizing the security of new browser releases,” Brown warned.
On the other hand, hackers are ramping up their assaults and coming up with inventive new attacks, so browsers whose vendors lag in issuing an update pose a security risk.
“Hopefully, this rapid release approach will also result in the faster patching of security vulnerabilities,” Brown remarked.
That’s exactly what Mozilla thinks.
“By releasing small, focused updates more often, we are able to deliver improved security and stability even as we introduce new features, which is better for our users, and for the Web,” Mozilla’s Nightingale said.
“If a serious security issue is found between regularly scheduled Firefox updates, we will release an interim update quickly, as we always have,” Nightingale stated.

Complete story here

Tommorows Cameras A Day Early: Lytro

lytro

have you ever taken a photo and only noticed it was out of focus afterwards? A new kind of camera from an ambitious startup could make such focusing mistakes a thing from the past. Able to take so-called “living photographs,” the Lytro camera captures images in such a way that viewers can change which part is in focus while viewing them.

The Lytro website gallery of “living” photos that shows off the light-field technology. With a click, you can re-focus the pictures on any part. The effect is impressive: a photo can take on entirely new meaning, depending on which part you focus on. Here is an example, a picture of two cats looks like a simple close-up when focused on the one in the foreground, but zeroing in on the background cat almost implies an ominous threat.

 

Google Results Are Now Instant!

Google Instant is a new search enhancement which shows results as you type. Google is pushing the limits of their technology and infrastructure to help everyuone get better search results, faster .Google’s key technical insight was that people type slowly, but read quickly, typically taking 300 milliseconds between keystrokes, but only 30 milliseconds to glance at another part of the page. This means that you can scan a results page while you type.

The most noticeable change is that you get to the right content much faster than before because you won’t have to finish typing your full search term, or even press “search.” Another shift is that seeing results as you type helps you formulate a better search term by providing instant feedback. You can now adapt your search on the fly until the results match exactly what you want. In time, we may wonder how search ever worked in any other way.

Voice Search

Use Google Chrome to speak your search queries instead of typing. Head to Google.com and click on the gray microphone in the search box.

Instant on Images

Google Instant is a search enhancement that shows results as you type. This experiment enables Instant on the Google Images search results page.

+1 button

Use +1 to give something your public stamp of approval, so friends, contacts, and others can find the best stuff when they search. Get recommendations for the things that interest you, right when you want them, in your search results.

Keyboard shortcuts

Navigate search results quickly and easily, minimizing use of your mouse.

Accessible View

Navigate search results quickly and easily, with just your keyboard. As you navigate, items are magnified for easier viewing. If you use a screen reader or talking browser, the relevant information is spoken automatically as you navigate.

 

Visit Google to opt-in

Your current experiment is no longer available. Try out a different one below. Note that you can only join ONE experiment at a time.

Try it out today http://www.google.com/experimental/index.html