Contents

1. 1 How to merge two lists, sort them and exclude repeated lines?
2. 2 How to find duplicates in two lists?
3. 3 Count the number of lines in a file and get only the number?
4. 4 String manipulation, matching, etc.
5. 5 Replace all slashes by commas in a file
6. 6 sed -e 's/\//,/g' results.csv > results-all.csv
7. 7 How to split file into several with the fixed length?
8. 8 How to add a character (or a string) at the end of each line of a file?
9. 9 How to add a character (or a string) at the beginning of each line of a file?
10. 10 How to rename a set of files or add an additional extension?
11. 11 How to add a fixed header at the top of each file in your dir?
12. 12 Convert 7-bit ASCII representations to UTF-8 Unicode
13. 13 Convert middle quotes in the four double quotes with single?
14. 14 Escape backslashes
15. 15 Convert middle quote in the three double quotes with single?
16. 16 How many lines are there in each file?
17. 17 Sed tutorial
18. 18 Awk tricks
19. 19 awk '{print $1,$2}' expected.constraints|tr ':' ' '|sort -k 3 -rn|column -t|lessOutput:name Person 0.9
20. 20 How to Read CSV or convert it to TSV using bash on Mac OS X?
21. 21 How to remove trailing and leading spaces from a string in awk?

How to merge two lists, sort them and exclude repeated lines?

cat file1name file2name | sort | uniq > outputfilename
this will sort ascending
to sort descending add -r option to sort:
cat file1name file2name | sort -r | uniq > outputfilename

Option 2:

sort file1name file2name | uniq -u > diffLines 

How to find duplicates in two lists?

comm -1 -2 <(sort first.txt) <(sort second.txt)

sed substitute command: match a regular expression and replace it by something else
sed s/"<www"/"<http:\/\/www"/ mappingbased_properties_en.nt >mappingbased_properties_en_fixed.nt
more
String matching in Perl

Replace all slashes by commas in a file

split -l 1000 file.nt
will split files into separate ones of lenth 1000

How to add a character (or a string) at the end of each line of a file?

add ">" at the end of each line ($ is a regex for end of the line):

sed 's/$/>/' myfile

This will not modify the file. To modify the file add option -i:

sed -i 's/$/>/' myfile

OR

save it to another file:

sed 's/$/>/' myfile > anotherfile

How to add a character (or a string) at the beginning of each line of a file?

add "<" at the beginning of each line (^ is a regex for end of the line):

sed 's/^/</' myfile

This will not modify the file. To modify the file add option -i:

sed -i 's/^/</' myfile

OR

save it to another file:

sed 's/^/</' myfile > anotherfile

How to rename a set of files or add an additional extension?

for i in *.*; do mv "$i" "$i.n3"; done

How to add a fixed header at the top of each file in your dir?

merge content of file1 with all other files in your dir (e.g. add a fixed header with file1 content at the top of each file in your dir):

for i in *.n3; do cat file1 "$i" > "$i.withhead.n3"; done

Convert 7-bit ASCII representations to UTF-8 Unicode

for i in *.n3; do ascii2uni "$i" > "$i.utf8.n3"; done

Convert middle quotes in the four double quotes with single?

e.g. if you have a string with ...."..."..."..".....

and need to convert it to:

...."...'...'..".....

do this:

for i in *.n3; do sed 's/\([\"].*\)[\"]\(.*\)[\"]\(.*[\"]\)/\1'\2'\3/g' "$i" > "$i.quotesfixed.n3"; done

Escape backslashes

for i in *.n3; do sed 's/\(.*\)[\\]\(.*\)/\1 \2/g' "$i" > "$i.bfixed.n3"; done

Convert middle quote in the three double quotes with single?

e.g. if you have a string with ...."..."..".....

and need to convert it to:

i.e. remove the middle one
for i in *.n3; do sed 's/\([\"].*\)[\"]\(.*[\"]\)/\1 \2/g' "$i" > "$i.3qfixed.n3"; done

How many lines are there in each file?

count numer of lines of the files with extension .txt in your specified dir:

find /thepathtothedir -maxdepth 1 -name "*.txt" -print0  | xargs -0 -n 1 wc -l

to put that list into a file use:
  find . -maxdepth 1 -name "*.txt" -print0  | xargs -0 -n 1 wc -l > lines.txt

Count the files with 0 lines:
  grep "^0 " lines.txt | wc -l

or with 10 lines:
  grep "^10" lines.txt  | wc -l

Sed tutorial

http://www.tutorialspoint.com/unix/unix-regular-expressions.htm

Awk tricks

awk '{print $1,$2}' expected.constraints|tr ':' ' '|sort -k 3 -rn|column -t|less

Output:

name Person 0.9

How to Read CSV or convert it to TSV using bash on Mac OS X?