Problem :
source is flat file which contains duplicates,we have 2 targets in the mapping,we need to pass unique records to one target & records which is repeating(duplicates) to another target.
can anyone please tell how we can achieve this.
Solutions 1:
Sort the data. Create a field in an Expression that counts the duplicate rows by comparing it to the previous row. Use a Router with two groups: where count=1 and where count >1.
Solution 2 :
put an aggregator transformation and do group by all ports and add a count(*) function in another output port and based on count(*)>1 or =1 you can use router to divert the distinct and duplicate records.
Solutions 3:
u can also pass all duplicate records.
by following these steps:
1. take sql qualifier with sorted input checked on.
2.pass all rows to aggregator , n do group by all ports, and take on port (count(*))
3. take another sql qualifier ,
4. use joiner with full outer join n join 2 & 3. withcondition (port matching)
5. take 2 filters to separate duplicate row & distinct rows.. based on condn: count(*)>1 n count(*)=1 and
pass it to respective targets.
source is flat file which contains duplicates,we have 2 targets in the mapping,we need to pass unique records to one target & records which is repeating(duplicates) to another target.
can anyone please tell how we can achieve this.
Solutions 1:
Sort the data. Create a field in an Expression that counts the duplicate rows by comparing it to the previous row. Use a Router with two groups: where count=1 and where count >1.
Solution 2 :
put an aggregator transformation and do group by all ports and add a count(*) function in another output port and based on count(*)>1 or =1 you can use router to divert the distinct and duplicate records.
Solutions 3:
u can also pass all duplicate records.
by following these steps:
1. take sql qualifier with sorted input checked on.
2.pass all rows to aggregator , n do group by all ports, and take on port (count(*))
3. take another sql qualifier ,
4. use joiner with full outer join n join 2 & 3. withcondition (port matching)
5. take 2 filters to separate duplicate row & distinct rows.. based on condn: count(*)>1 n count(*)=1 and
pass it to respective targets.
No comments:
Post a Comment