Unix Commands ..

Today we will practice some Unix commands which usually we use and are really helpful and required when your file size is more than 60K records, because those files you can not open at Excel.
1. How you will find the row count of a file?
2. How you will find total number of files inside any directory?
3. How will you print first 2 lines of file?
4. How to find number of records without displaying file name in output
5. Suppose you have 100 records, out of which you want to read 10 to 20 records. You have a header in a file, how will you do that?
6. I want distinct Department_Id from Employee File
7. Check with Piyush what command he used to fire for record count without the header record.
8. I want all Employees with department as 10 ?
9. I've a flat file where 1st two bytes of each record is an identifier for that record. How do I find distinct count of these records?
10. How will you compare 2 files
11. Find out files starting with Emp in a particular directory
12. How will you find particular name in a file ?
13. How will you find files in a directory ?
14. How will you search file in same / current directory?
15. How will you change the file permissions of all files inside particular directory ?
16. How will you search all files owned by a particular user
17. How will you replace character / string in a particular file with another character / string ?
18. How will you sort files by their size in a particular direcotry?
19. How will you sort files by their timestamps
20. To find out total size of a folder
21. How will you delete particular content of a file
22. How will you print last 2 lines of a file
23. How to find folders in particular directory

1. How you will find the row count of a file?
Mandar@Mandar-PC /MyFiles
$ wc -l Employees.csv
108 Employees.csv

2. How you will find total number of files inside any directory?
Mandar@Mandar-PC /MyFiles
$ ls | wc -l
8

3. How will you print first 2 lines of file?
Mandar@Mandar-PC /MyFiles
$ head -2 Employees.csv
employee_id,first_name,last_name,email,phone_number,hire_date,job_id,salary,commission_pct,manager_id,department_id
100,Steven,King,SKING,515.123.4567,6/17/1987,AD_PRES,24000,null,null,90


4. How to find number of records without displaying file name in output
Mandar@Mandar-PC /MyFiles
$ awk '{n++} END {print n}' Employees.csv
108


Even this will work same way Mandar@Mandar-PC /MyFiles
$ wc -l < Employees.csv
108

5. Suppose you have 100 records, out of which you want to read 10 to 20 records. You have a header in a file, how will you do that?
Mandar@Mandar-PC /MyFiles
$ head -20 Employees.csv | tail -10
109,Daniel,Faviet,DFAVIET,515.124.4169,8/16/1994,FI_ACCOUNT,9000,null,108,100
110,John,Chen,JCHEN,515.124.4269,9/28/1997,FI_ACCOUNT,8200,null,108,100
111,Ismael,Sciarra,ISCIARRA,515.124.4369,9/30/1997,FI_ACCOUNT,7700,null,108,100
112,Jose Manuel,Urman,JMURMAN,515.124.4469,3/7/1998,FI_ACCOUNT,7800,null,108,100
113,Luis,Popp,LPOPP,515.124.4567,12/7/1999,FI_ACCOUNT,6900,null,108,100
114,Den,Raphaely,DRAPHEAL,515.127.4561,12/7/1994,PU_MAN,11000,null,100,30
115,Alexander,Khoo,AKHOO,515.127.4562,5/18/1995,PU_CLERK,3100,null,114,30
116,Shelli,Baida,SBAIDA,515.127.4563,12/24/1997,PU_CLERK,2900,null,114,30
117,Sigal,Tobias,STOBIAS,515.127.4564,7/24/1997,PU_CLERK,2800,null,114,30
118,Guy,Himuro,GHIMURO,515.127.4565,11/15/1998,PU_CLERK,2600,null,114,30


6. I want distinct Department_Id from Employee File
Mandar@Mandar-PC /MyFiles
$ cut -d"," -f11 Employees.csv | sort | uniq
10
100
110
20
30
40
50
60
70
80
90
department_id
null

Give man cut so that you will get all parameter informationuniq will output all lines exactly once:
uniq -d will output all lines that appear more than once, and it will print them once:
uniq -u will output all lines that appear exactly once, and it will print them once


7. Check with Piyush what command he used to fire for record count without the header record.


8. I want all Employees with department as 10 ?
Mandar@Mandar-PC /MyFiles/NewFolder
$ awk -F"," '{ if($11=10) print $0}' Employees1.csv > Dep10File.csv


9.I've a flat file where 1st two bytes of each record is an identifier for that record. How do I find distinct count of these records?
Flat file ex:
AB|123|hello moto|
AB|456|googly|
BC|sick day|booya|
ID|inter doing|hahahah

So I want to find distinct record count... it should give me
AB 2
BC 1
ID 1

Mandar@Mandar-PC /MyFiles
$ awk -F'|' '{++c[$1]}END{for (i in c)print i, c[i]}' SampleData.txt
AB 2
BC 1
ID 1


10. How will you compare 2 files
Mandar@Mandar-PC /MyFiles
$ comp mandar.txt myname.txt
Comparing mandar.txt and myname.txt...
Compare error at OFFSET 6
file1 = A
file2 = 20
Compare error at OFFSET 10
file1 = A
file2 = 20
Compare more files (Y/N) ? n

Mandar@Mandar-PC /MyFiles
$ diff mandar.txt myname.txt
1,3c1
< mandar
< raghunath
< gogate
---
> mandar raghunath gogate

Mandar@Mandar-PC /MyFiles
$ sdiff mandar.txt myname.txt
mandar | mandar raghunath gogate
raghunath <
gogate <


11. Find out files starting with Emp in a particular directory
Mandar@Mandar-PC /MyFiles
$ ls -l | grep Emp*
-rwxr-xr-x 1 Mandar None 8314 May 1 00:39 Employees.csv


12. How will you find particular name in a file ?
Mandar@Mandar-PC /MyFiles
$ grep Sundi* Employees.csv
166,Sundar,Ande,SANDE,011.44.1346.629268,3/24/2000,SA_REP,6400,0.1,147,80
173,Sundita,Kumar,SKUMAR,011.44.1343.329268,4/21/2000,SA_REP,6100,0.1,148,80


13. How will you find files in a directory ?
Mandar@Mandar-PC /MyFiles
$ find -name 'Emp*'
./Employees.csv
./Employees1.csv
./Employees10.csv
./Employees11.csv
./Employees12.csv
./Employees2.csv
./Employees3.csv
./Employees4.csv
./Employees5.csv
./Employees6.csv
./Employees7.csv
./Employees8.csv
./Employees9.csv

As files were inside MyFile folder itself, it gave the search result like this. but same command works for finding files in sub - direcotry also. Now I have moded all Empl* files to new directory called NewFolder which is inside MyFiles directory
Mandar@Mandar-PC /MyFiles
$ find -name 'Emp*'
./NewFolder/Employees.csv
./NewFolder/Employees1.csv
./NewFolder/Employees10.csv
./NewFolder/Employees11.csv
./NewFolder/Employees12.csv
./NewFolder/Employees2.csv
./NewFolder/Employees3.csv
./NewFolder/Employees4.csv
./NewFolder/Employees5.csv
./NewFolder/Employees6.csv
./NewFolder/Employees7.csv
./NewFolder/Employees8.csv
./NewFolder/Employees9.csv

This command searches even the directory name and not only the file name
Mandar@Mandar-PC /MyFiles
$ find . -name New*
./NewFolder

To find only the files and exclude directories Mandar@Mandar-PC /MyFiles
$ find . -type f -name New*

14. How will you search file in same / current directory?
To search in all directories Mandar@Mandar-PC /MyFiles
$ find / -type f -name Emp*

15. How will you change the file permissions of all files inside particular directory ?
Mandar@Mandar-PC /MyFiles
$ find ./NewFolder/ -exec chmod 777 {} \;

16. How will you search all files owned by a particular user
Mandar@Mandar-PC /MyFiles
$ find . -user mandar

17. How will you replace character / string in a particular file with another character / string ?
Mandar@Mandar-PC /MyFiles/NewFolder
$ less Employees1.csv | tr 'manager_id' 'MANAGER_ID' | less
Above command will give display character with MANAGER_ID at command line You can redirect this output to new file with below command
Mandar@Mandar-PC /MyFiles/NewFolder
$ less Employees1.csv | tr 'manager_id' 'MANAGER_ID' > MyNewFile.csv

18. How will you sort files by their size in a particular direcotry?
Mandar@Mandar-PC /MyFiles/NewFolder
$ du -s * | sort -n
0 Employees.csv
12 Employees1.csv
12 MyNewFile.csv
20 Employees2.csv
36 Employees3.csv
68 Employees4.csv
180 Employees5.csv
344 Employees6.csv
764 Employees7.csv
2292 Employees8.csv
9160 Employees9.csv
27476 Employees10.csv
137376 Employees11.csv
412128 Employees12.csv

Mandar@Mandar-PC /MyFiles/NewFolder
$ du -a | sort -n
0 ./Employees.csv
1 ./helloworld.txt
12 ./Employees1.csv
12 ./MyNewFile.csv
20 ./Employees2.csv
36 ./Employees3.csv
68 ./Employees4.csv
180 ./Employees5.csv
344 ./Employees6.csv
764 ./Employees7.csv
2292 ./Employees8.csv
9160 ./Employees9.csv
27476 ./Employees10.csv
137376 ./Employees11.csv
412128 ./Employees12.csv
589873 .


19. How will you sort files by their timestamps
Mandar@Mandar-PC /MyFiles/NewFolder
$ ls -ctl | sort -n
-rw-r--r-- 1 Mandar None 11 May 5 12:34 helloworld.txt
-rw-r--r-- 1 Mandar None 8314 May 5 12:20 MyNewFile.csv
-rwxrwxrwx 1 Mandar None 0 May 5 12:09 Employees.csv
-rwxrwxrwx 1 Mandar None 8314 May 5 12:03 Employees1.csv
-rwxrwxrwx 1 Mandar None 16628 May 5 12:03 Employees2.csv
-rwxrwxrwx 1 Mandar None 33256 May 5 12:03 Employees3.csv
-rwxrwxrwx 1 Mandar None 66512 May 5 12:03 Employees4.csv
-rwxrwxrwx 1 Mandar None 182908 May 5 12:03 Employees5.csv
-rwxrwxrwx 1 Mandar None 349188 May 5 12:03 Employees6.csv
-rwxrwxrwx 1 Mandar None 781516 May 5 12:03 Employees7.csv
-rwxrwxrwx 1 Mandar None 2344548 May 5 12:03 Employees8.csv
-rwxrwxrwx 1 Mandar None 9378192 May 5 12:03 Employees9.csv
-rwxrwxrwx 1 Mandar None 28134576 May 5 12:03 Employees10.csv
-rwxrwxrwx 1 Mandar None 140672880 May 5 12:03 Employees11.csv
-rwxrwxrwx 1 Mandar None 422018640 May 5 12:03 Employees12.csv
total 589869


20. To find out total size of a folder
Mandar@Mandar-PC /MyFiles/NewFolder
$ du -h
577M .
22. How will you print last 2 lines of a file
Mandar@Mandar-PC /MyFiles
$ tail -2 Second.sh
echo -e "\n\nFile $1, found and successfully echoed"
fi

23. How to find folders in particular directory
Mandar@Mandar-PC /MyFiles
$ find . -type d
.
./NewFolder
Posted by Man