Posted: Wed Oct 29, 2003 10:44 am Post subject: Synsort Exit Rtn
Here is my situation:
I am concatenating 5 datasets as input. Each is of the same format.
Header record
Detail record
Trailer record
I need to put these through a syncsort that will sum all duplicate detail records based on positions 1-63. The final result will be one header record, sort/summed detail reocrds, and a trailer record with updated totals. This will look like
I am sort/summing based on a key. Records that match will get combined, and dollars in money buckets(154,9,PD,163,9,PD,172,9,PD) will be added together.
My problem is that for the specified key, header and trailer records are exactly the same. Thus it wants to combine them and sum on the money fields. Since there are no money fields on the header record, and the money fields on the trailer record are in a different place, this results in a SOC7.
Solution so far
1. Omit header and control records in the syncsort, and then create new ones in the exit routine. This works sometimes. But sometimes the header record will contain information i cannot replicate once its deleted. For example, sequence number, date created, etc. To account for that I used solution #2.
2. Pull off header reocrds as needed before synsort. Sort/sum detail records and then merge with cycle record pulled off in previous step.
Either way i know so far involves extra coding, or extra JCL steps. Just wondering if anyone knows of an easier way to get around this.
Joined: 26 Nov 2002 Posts: 12378 Topics: 75 Location: San Jose
Posted: Wed Oct 29, 2003 11:55 am Post subject:
Disco_stu,
I really don't see the use of an exit in here unless you are doing some complex programming logic. Every thing you asked for can be done using sort with just one pass.The simplest and easiest solution(option 2) would be to pull off the header and trailer records and then sum sort on the detail records. Ofcourse this would involve an extra pass of the data.
There is a way to do all in just one pass. I need the following details.
1. How do you distinguish a header record, a detail record and the trailer record?
2. what are the contents in the positions(154, 163 & 172 ) on the header and trailer record?
3. where does the money field start on the trailer record as you said it is at a different position
4. what is the recfm,lrecl of the files?
5. Please post a sample of input records and the desired output
Btw I don't think you can sum on 9 bytes on packed decimal field.Sort can handle a PD length up to 8 (15 digits and a sign), so if the length is really 9 (17 digits and a sign), that's a problem, unless the first byte of the PD field is always X'00'.
Let me know these details and I will post the solution.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Oct 29, 2003 12:26 pm Post subject:
Kolusu said
Quote:
I don't think you can sum on 9 bytes on packed decimal field. Sort can handle a PD length up to 8 (15 digits and a sign)
Depends on which "Sort" you mean - DFSORT can handle a PD length up to 16 bytes for SUM. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
<b>I really don't see the use of an exit in here unless you are doing some complex programming logic. </b>
I need to update the trailer record with the new record count. if records will end up being combined, the count will decrease. I also want to merge all the trailer records into one. I dont want the final output to have more than one header or trailer record. Maybe there is an easier way to do this...
<b>1. How do you distinguish a header record, a detail record and the trailer record? </b>
header - 'CC' in pos 1-2
detail - 'DL' in pos 1-2
trailer - 'XX' in pos 1-2
<b>2. what are the contents in the positions(154, 163 & 172 ) on the header and trailer record? </b>
header - spaces
trailer - various money buckets, but the positions arent the same as the detail reocrd, so they dont match up with the detail layout, hence soc7
<b>3. where does the money field start on the trailer record as you said it is at a different position</b>
start at positions 162,171,180; each defined as PIC S9(15)V99 comp-3
<b>4. what is the recfm,lrecl of the files?</b>
293 bytes, fixed block
<b>5. Please post a sample of input records and the desired output </b>
I'll use an example here. i figured real data would be too big. I have 2 input datasets here. each has a header (CC) record, a few detail (DL) records, and a trailer (XX) record.
The header record is all spaces.
The deail reocord is made up of:
Positions 3-5 are my key.
At the end, 13-15 is a money field.
Trailer record has a record count in positions 3-5 and a total money field in positions 10-12.
Dataset 1
CC
DLAAABBBBBBB001
DLAAABBBBBBB000
DLAAABBBBBBB003
XX003AAAA004AAA
Dataset 2
CC
DLAAABBBBBBB002
DLAAABBBBBBB002
DLCCCBBBBBBB002
XX003AAAA006AAA
Output
CC
DLAAABBBBBBB008
DLCCCBBBBBBB002
XX002AAAA010AAA
All recs with key 'AAA' are combined. New trailer record with updated record count and money amt is created.
<b>Btw I don't think you can sum on 9 bytes on packed decimal field.Sort can handle a PD length up to 8 (15 digits and a sign), so if the length is really 9 (17 digits and a sign), that's a problem, unless the first byte of the PD field is always X'00'. </b>
See above for field definition. It's been working so far. but since the field is so big, the first byte is probably always X'00', like you said. I dont really have any freedom to change the field size.
<b>Depends on which "Sort" you mean - DFSORT can handle a PD length up to 16 bytes for SUM.</B>
Joined: 26 Nov 2002 Posts: 12378 Topics: 75 Location: San Jose
Posted: Wed Oct 29, 2003 2:16 pm Post subject:
Disco_stu,
The following JCL will give you the desired results. The only thing I was not able to do is the updation of the trailer record count.A brief explanation of the JOb.
I am using the change parm to change the contents of fields for header and trailer records.For every header & trailer record we are going to change the bytes 154, 163, 172 to have zero in packed decimal format.
I am also adding another 3 fields for which I am using the change operator to pad zeroes for every header & detail record. But for the trailer record I copy the contents from the bytes 162,171,180.
Now I summing the contents in 154, 163, 172, as well as the contents which we added i.e bytes 294,303,312
Now all the detailed records will be summed in the bytes 154 thru 180 and the trailer records will be summed in the bytes 294 thru 320.
Now using outrec fields I am adding the summed value in 155 bytes with the value in 295 bytes. similarly for the values in 164 bytes with the value in 304 .similary for the values in 173 with the value in 313. I am adding on 8 bytes as Sort can handle a PD length up to 8 and I padded a X'00' in front to the fields.
Now using change parm on outfil outrec I am padding spaces for the header record.
I haven't tested this code as I don't have the time to set up the input data.But you get the idea as to how it is being done.
Joined: 26 Nov 2002 Posts: 12378 Topics: 75 Location: San Jose
Posted: Wed Oct 29, 2003 2:22 pm Post subject:
Frank,
Thanks for pointing that for summation DFSORT can handle upto 16 bytes of PD field.It is the same for syncsort also.But the max length on inrec/outrec fields is 8 for PD for syncsort.Does DFSORT allow more than 8 bytes for PD field on outrec?? I get a numeric field error if try to convert a zoned decimal to PD field with length of 9 bytes.
Thanks. There's alot of stuff in there I've never seen before. I'm gonna have to go through it and figure out the details. But from your description i get the basic idea.
Unfortunately, the count definitely needs to get updated. Otherwise i'll get out of balance errors down the line.
But you definitly exposed me to some new synsort code and some new ideas, and i think i'm going to be able to apply some of the things i see here to other processes, so thank you for your time.
Last edited by Disco_Stu on Wed Oct 29, 2003 3:27 pm; edited 1 time in total
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Oct 29, 2003 3:18 pm Post subject:
Kolusu said
Quote:
the max length on inrec/outrec fields is 8 for PD for syncsort. Does DFSORT allow more than 8 bytes for PD field on outrec??
No, that limit is 8 for DFSORT as well. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
From what i know, they seem to be very similar. So I'm just curious as to why you did what you did. could those outrec's been combined? or does this essentially allow you to process an outgoing record twice?
Joined: 26 Nov 2002 Posts: 12378 Topics: 75 Location: San Jose
Posted: Wed Oct 29, 2003 4:35 pm Post subject:
Disco_Stu,
Yes I am reformating the output twice. I need to process the output record twice as I want sum up the quantities and then also changes the value to spaces on the header record.
Basically this is what happens after summing of the records
Code:
TYPE MONEY1 MONEY2 MONEY3 TEMP1 TEMP2 TEMP3
(1-2) (155-162) (164-171) (173 - 180) (295-302) (304-311) (313-320)
CC 0 0 0 0 0 0
DL A 1000 2000 3000 0 0 0
DL B 1300 2200 300 0 0 0
...
XX 0 0 0 4000 5000 6000
Now the first outrec fields will sum
Money1 with TEMP1
Money2 with TEMP2
Money3 with TEMP3
Now the outrec on the outfil statements takes these updated values and changes the zeroes on the header record to have spaces.Basically we are processing the output record twice.
To get the updated record you can use the trailer feature on the outfil and it will give you the new count of records which is inclusive of the header and trailer record.
Hope this explanation is clear
Kolusu
Last edited by kolusu on Wed Oct 29, 2003 5:49 pm; edited 1 time in total
Adjusting the LRECL isnt really an option. The count is in positin 23 for 4 bytes. That is spaces on the header record, and overlaps a couple alphanumeric fields on the detail record.
Joined: 26 Nov 2002 Posts: 12378 Topics: 75 Location: San Jose
Posted: Thu Oct 30, 2003 10:21 am Post subject:
Disco_stu,
If the alphanumeric fileds in position 23 for 4 bytes on the header and detail records are not that important then you can add a seqnum to outrec fields and in the end subtract 2 from it so that you will get the record count.
Just incase if you are willing to increase the LRECL, then you add the seqnum at the end so that you will keep all the values(pos 23 - 26) of header and detail records intact.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum