Solve real-world shell scripting problems with over 110 simple but incredibly effective recipes

Sarath Lakshman

Lectura muy recomendada, sobre las cosas que podemos hacer con bash.

Pongo unos ejemplos que he destacado:

Modificar la extensión de los archivos de JPG a jpg

rename *.JPG *.jpg

Moficar según una expresión regular

rename 's/ /_/g' *

Buscar resultados en un diccionario

#!/bin/bash
#Filename: aspellcheck.sh
word=$1
output=`echo \"$word\" | aspell list`
if [ -z $output ]; then
echo $word is a dictionary word;
else
echo $word is not a dictionary word;
fi

Presentar la frecuencia de las palabras de un archivo

#!/bin/bash
#Name: word_freq.sh
#Description: Find out frequency of words in a file
if [ $# -ne 1 ];
then
echo "Usage: $0 filename";
exit -1
fi
filename=$1
egrep -o "\b[[:alpha:]]+\b" $filename | \
awk '{ count[$0]++ }
END{ printf("%-14s%s\n","Word","Count") ;
for(ind in count)
{ printf("%-14s%d\n",ind,count[ind]); }
}'

Matched string notation (&)

In sed we can use & as the matched string for the substitution pattern such that we can use
the matched string in replacement string.
For example:

$ echo this is an example | sed 's/\w\+/[&]/g'

[this] [is] [an] [example]
Here the regex \w\+ matches every word. Then we replace it with [&]. & corresponds to the
word that is matched.

Substring match notation (\1)

& is a string which corresponds to match string for the given pattern. But we can also match
the substrings of the given pattern. Let's see how to do it.

$ echo this is digit 7 in a number | sed 's/digit \([0-9]\)/\1/'

this is 7 in a number
It replaces digit 7 with 7. The substring matched is 7. \(pattern\)is used to match the
substring. The pattern is enclosed in () and is escaped with slashes. For the first substring
match, the corresponding notation is \1, for the second it is \2, and so on. Go through the
following example with multiple matches:
$ echo seven EIGHT | sed 's/\([a-z]\+\) \([A-Z]\+\)/\2 \1/'
EIGHT seven
([a-z]\+\) matches the first word and \([A-Z]\+\)matches the second word. \1 and
\2 are used for referencing them. This type of referencing is called back referencing. In the
replacement part, their order is changed as \2 \1 and hence it appears in reverse order.

Awk avanzado

$ seq 5 | awk 'BEGIN{ sum=0; print "Summation:" }
{ print $1"+"; sum+=$1 } END { print "=="; print sum }'

Passing a variable value from outside to awk
By using the -v argument, we can pass external values (other than from stdin) to awk
as follows:

$ VAR=10000
$ echo | awk -v VARIABLE=$VAR'{ print VARIABLE }'

1
There is a flexible alternate method to pass many variable values from outside awk.
For example:

$ var1="Variable1" ; var2="Variable2"
$ echo | awk '{ print v1,v2 }' v1=$var1 v2=$var2

Variable1 Variable2
When input is given through a file rather than standard input, use:

$ awk '{ print v1,v2 }' v1=$var1 v2=$var2 filename

In the above method, variables are specified as key-value pairs separated by space
(v1=$var1 v2=$var2) as command arguments to awk soon after the BEGIN, { } and END
blocks.

Compressing or decompressing JavaScript

$ catsample.js |
tr -d '\n\t' |
\
tr -s ' ' \
| sed 's:/\*.*\*/::g' \
| sed 's/ \?\([{}();,:]\) \?/\1/g'

The output is as follows:

functionsign_out(){$("#loading").show();$.get("log_
in",{logout:"True"},function(){window.location="";});}

Let's write a decompression script for making obfuscated code readable as follows:

$ cat obfuscated.txt | sed 's/;/;\n/g; s/{/{\n\n/g; s/}/\n\n}/g'

Acceder a gmail desde linea de comandos

#!/bin/bash
Filename: fetch_gmail.sh
#Description: Fetch gmail tool
username="PUT_USERNAME_HERE"
password="PUT_PASSWORD_HERE"
SHOW_COUNT=5 # No of recent unread mails to be shown
echo
curl -u $username:$password --silent "https://mail.google.com/mail/
feed/atom" | \
tr -d '\n' | sed 's:</entry>:\n:g' |\
sed 's/.*<title>\(.*\)<\/title.*<author><name>\([^<]*\)<\/
name><email>
\([^<]*\).*/Author: \2 [\3] \nSubject: \1\n/' | \
head -n $(( $SHOW_COUNT * 3 ))

Bajar todas las imágenes de una pagina web

#!/bin/bash
#Description: Images downloader
#Filename: img_downloader.sh
if [ $# -ne 3 ];
then
echo "Usage: $0 URL -d DIRECTORY"
exit -1
fi
for i in {1..4}
do
case $1 in
-d) shift; directory=$1; shift ;;
*) url=${url:-$1}; shift;;
esac
done
mkdir -p $directory;
baseurl=$(echo $url | egrep -o "https?://[a-z.]+")
curl –s $url | egrep -o "<img src=[^>]*>" |
sed 's/<img src=\"\([^"]*\).*/\1/g' > /tmp/$$.list
sed -i "s|^/|$baseurl/|" /tmp/$$.list
cd $directory;
while read filename;
do
curl –s -O "$filename" --silent
done < /tmp/$$.list

Twiter desde consola

#!/bin/bash
#Filename: tweets.sh
#Description: Basic twitter client
USERNAME="PUT_USERNAME_HERE"
PASSWORD="PUT_PASSWORD_HERE"
COUNT="PUT_NO_OF_TWEETS"
if [[ "$1" != "read" ]] && [[ "$1" != "tweet" ]];
then
echo -e "Usage: $0 send status_message\n
OR\n
exit -1;
fi
if [[ "$1" = "read" ]];
then
curl --silent -u $USERNAME:$PASSWORD
friends_timeline.rss | \
grep title | \
tail -n +2 | \
head -n $COUNT | \
sed 's:.*<title>\([^<]*\).*:\n\1:'
$0 read\n"
http://twitter.com/statuses/
elif [[ "$1" = "tweet" ]];
then
status=$( echo $@ | tr -d '"' | sed 's/.*tweet //')
curl --silent -u $USERNAME:$PASSWORD -d status="$status" http://
twitter.com/statuses/update.xml > /dev/null
echo 'Tweeted :)'
fi

define: de google desde consola

#!/bin/bash
#Filename: define.sh
#Description: A Google define: frontend
limit=0
if [ ! $# -ge 1 ];
then
echo -e "Usage: $0 WORD [-n No_of_definitions]\n"
exit -1;
fi
if [ "$2" = "-n" ];
then
limit=$3;
let limit++
fi
word=$1
lynx -dump http://www.google.co.in/search?q=define:$word | \
awk '/Defini/,/Find defini/' | head -n -1 | sed 's:*:\n*:; s:^[ ]*::'
| \
grep -v "[[0-9]]" | \
awk '{
if ( substr($0,1,1) == "*" )
{ sub("*",++count".") } ;
print
} ' > /tmp/$$.txt
echo
if [ $limit -ge 1 ];
then
cat /tmp/$$.txt | sed -n "/^1\./, /${limit}/p" | head -n -1
else
cat /tmp/$$.txt;
fi

Buscando enlaces rotos de una web

#!/bin/bash
#Filename: find_broken.sh
#Description: Find broken links in a website
if [ $# -eq 2 ];
then
echo -e "$Usage $0 URL\n"
exit -1;
fi
echo Broken links:
mkdir /tmp/$$.lynx
cd /tmp/$$.lynx
lynx -traversal $1 > /dev/null
count=0;
sort -u reject.dat > links.txt
while read link;
do
output=`curl -I $link -s | grep "HTTP/.*OK"`;
if [[ -z $output ]];
then
echo $link;
let count++
fi
done < links.txt
[ $count -eq 0 ] && echo No broken links found.

ssh con compresión

ssh -C user@host

¿ssh con zenity?

$ ssh username@remotehost 'export DISPLAY=:0 ; zenity --info --text "This
is a message"'

Lanzar un comando cada x segundos

watch -n 1 -d 'ls -l /tmp'

Mysqlcon bash

#!/bin/bash
#Filename: read_db.sh
#Description: Read from the database
USER="user"
PASS="user"
depts=`mysql -u $USER -p$PASS students <<EOF | tail -n +2
SELECT DISTINCT dept FROM students;
EOF`
for d in $depts;
do
echo Department : $d
result="`mysql -u $USER -p$PASS students <<EOF
SET @i:=0;
SELECT @i:=@i+1 as rank,name,mark FROM students WHERE dept="$d" ORDER
BY mark DESC;
EOF`"
echo "$result"
echo
done