This is an old revision of the document!

BASH - Files - Removing Duplicate Lines from Files

#! /bin/sh
 
echo -n "Enter Filename-> "
read filename
if [ -f "$filename" ]; then
sort $filename | uniq | tee sorted.txt
else
echo "No $filename in $pwd...try again"
fi
 
exit 0

NOTE: The above script goes line by line through your file and removes any duplicate line.

It then places the new content into a new file and keeps the original file intact.