Eliminating Duplicate Rows from Flat File

[cyrus]

In Informatica, how is it done to eliminate duplicated rows from a flat file?

[zsk_00]

One way is to use aggregator. Group by all the columns and the output will not have any duplicates. If the input file size is big, use sorter before aggregator and check the [sorted input] box in the aggregator (this is for performace improvement)

Zafar

[a.joycy]

Inside UNIX you can use this commande to eliminate duplicated rows from a flat file"sed '$!N; /^\(.*\)\n\1$/!P; D'"

Ex: cat file sed '$!N; /^\(.*\)\n\1$/!P; D' > /home/tmp.dat
cat /home/tmp.dat > file
rm /home/tmp.dat