Here i need only the first record to be present in the output file 2(as all the records would have same data on position 509(length=5)) _________________ Rasprasad S
Joined: 26 Nov 2002 Posts: 12369 Topics: 75 Location: San Jose
Posted: Fri Dec 13, 2002 5:35 am Post subject:
Rasprasad,
The following jcl will give you the desired results. you cannot code sum fields statement on outfil. so initially we copy the out2 file to a temp file and then sort it to remove the duplicates.
Joined: 02 Dec 2002 Posts: 20 Topics: 0 Location: Yorkshire, England
Posted: Fri Dec 13, 2002 6:24 am Post subject:
Kolusu,
I had come up with the following:
Code:
//TOOLIN DD *
SORT FROM(IN) TO(OUT1) USING(CTL1)
SORT FROM(IN) TO(OUT2) USING(CTL2)
/*
//CTL1CNTL DD *
SORT FIELDS=(509,5,PD,A)
INCLUDE COND=(35,3,CH,EQ,C'010')
/*
//CTL2CNTL DD *
SORT FIELDS=(509,5,PD,A)
SUM FIELDS=NONE
INCLUDE COND=(35,3,CH,EQ,C'020')
/*
My knowledge of SYNCTOOL is basically what I have picked up from your posts, so could you comment on whether my solution is okay/efficient etc.
I suspect that my solution may involve two passes of the file, so would affect run times on a large input file. _________________ Cheers - Mike
Joined: 26 Nov 2002 Posts: 12369 Topics: 75 Location: San Jose
Posted: Fri Dec 13, 2002 6:38 am Post subject:
Mike,
Techinically speaking your solution is okay. But efficiency wise it is not. The second pass is doing a sum sort on the entire file instead of the specific records.
Let us assume that the input file have 1 million records and 800,000 records are having '10' in postion 35 and only 200,000 records are having '20' in position 35.Out of these 200,000 only 50,000 are duplicates.
So if we split the file initially into 2 different files and then sumsort on the smaller files.
Your solution will sum sort on the entire file and it affects the performance.
For relatively small files you wont even notice the difference.
Even in my posted solution , the first step can be a COPY instead of sort. But I assumed rasprasad wanted to have both files sorted on field at 509.
I hope I explained it clearly.Let me know if you have any questions
Joined: 26 Nov 2002 Posts: 12369 Topics: 75 Location: San Jose
Posted: Fri Dec 13, 2002 6:55 am Post subject:
Mike,
Well this another version to implement your logic. we first sumsort the entire file to eliminate the duplicates, but we will be writting the duplicates to SORTXSUM file.The XSUM parameter of the SUM statement sends all records deleted to the //SORTXSUM DD.
The second step takes in the SORTXSUM file and includes only '10' records and appends the data to out1 file
The second step would then remove duplicates from the SORTOF02.
Code:
//SYSIN DD *
SORT FIELDS=(509,5,PD,A)
SUM FIELDS=NONE
/*
I suppose the most efficient version depends on the expected data.
N.B. It should be noted for DFSORT users that XSUM is only available in SYNCSORT, as I have discovered when trying to 'advise' users of that product. _________________ Cheers - Mike
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Fri Dec 13, 2002 11:14 am Post subject:
Mike said "It should be noted for DFSORT users that XSUM is only available in SYNCSORT, as I have discovered when trying to 'advise' users of that product".
Mike,
Although DFSORT does not support XSUM, it does support the same function (and more) through ICETOOL's SELECT with DISCARD feature (which Syncsort does not support). For more information on that, see the "Keep dropped duplicate records (XSUM)" Smart DFSORT Trick at:
Joined: 02 Dec 2002 Posts: 20 Topics: 0 Location: Yorkshire, England
Posted: Fri Dec 13, 2002 11:27 am Post subject:
Frank,
rest assured that I was merely making the point that XSUM is a SYNCSORT only statement, in the context of the solutions given in this (SYNCSORT) thread.
I am certainly not qualified to talk about the alternative options in DFSORT as I do not have access to your product. _________________ Cheers - Mike
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Fri Dec 13, 2002 1:45 pm Post subject:
Mike,
I guess I misinterpreted what you said.
At any rate, even if your site doesn't have access to a DFSORT license, you still have access to the online DFSORT books in case you're ever curious about DFSORT/ICETOOL/ICEGENER.
And if anybody is interested in the DFSORT Team's analysis of DFSORT's advantages, contact me offline (yaeger@us.ibm.com) and I'll send you a document on that. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum