MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Synsort Exit Rtn

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
Disco_Stu
Beginner


Joined: 04 Feb 2003
Posts: 19
Topics: 7

PostPosted: Wed Oct 29, 2003 10:44 am    Post subject: Synsort Exit Rtn Reply with quote

Here is my situation:

I am concatenating 5 datasets as input. Each is of the same format.


    Header record
    Detail record
    Trailer record


I need to put these through a syncsort that will sum all duplicate detail records based on positions 1-63. The final result will be one header record, sort/summed detail reocrds, and a trailer record with updated totals. This will look like

Code:

//STEP02A  EXEC PGM=SYNCSORT,PARM='MSG=AP,VSIO=NONE,E35=COB'         
//MODLIB   DD DSN=ENDH1.B06CL003.SBATLOAD,DISP=SHR                   
//SORTIN   DD DSN=INPUTDATASET1,DISP=SHR                         
//         DD DSN=INPUTDATASET2,DISP=SHR     
//         DD DSN=INPUTDATASET3,DISP=SHR 
//         DD DSN=INPUTDATASET4,DISP=SHR 
//         DD DSN=INPUTDATASET5,DISP=SHR                     
//*                                                                 
//SORTOUT  DD DSN=OUTPUTDATASET,             
//            SPACE=(CYL,(25,25),RLSE),                             
//            DISP=(NEW,CATLG,DELETE),                               
//            UNIT=TPUB,                                             
//            DCB=(SYS2.GDG,RECFM=FB,LRECL=293,BLKSIZE=0,BUFNO=20)   
//*                                                                 
//SYSIN    DD *     
  SORT FIELDS=(1,63,CH,A),                 
  SIZE=E99999999,DYNALLOC=(SYSDA,32)                   
  SUM FIELDS=(154,9,PD,163,9,PD,172,9,PD) 
  MODS E35=(Z7476547,100,MODLIB,N)           


I am sort/summing based on a key. Records that match will get combined, and dollars in money buckets(154,9,PD,163,9,PD,172,9,PD) will be added together.

My problem is that for the specified key, header and trailer records are exactly the same. Thus it wants to combine them and sum on the money fields. Since there are no money fields on the header record, and the money fields on the trailer record are in a different place, this results in a SOC7.

Solution so far

1. Omit header and control records in the syncsort, and then create new ones in the exit routine. This works sometimes. But sometimes the header record will contain information i cannot replicate once its deleted. For example, sequence number, date created, etc. To account for that I used solution #2.

2. Pull off header reocrds as needed before synsort. Sort/sum detail records and then merge with cycle record pulled off in previous step.


Either way i know so far involves extra coding, or extra JCL steps. Just wondering if anyone knows of an easier way to get around this.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Wed Oct 29, 2003 11:55 am    Post subject: Reply with quote

Disco_stu,

I really don't see the use of an exit in here unless you are doing some complex programming logic. Every thing you asked for can be done using sort with just one pass.The simplest and easiest solution(option 2) would be to pull off the header and trailer records and then sum sort on the detail records. Ofcourse this would involve an extra pass of the data.

There is a way to do all in just one pass. I need the following details.

1. How do you distinguish a header record, a detail record and the trailer record?

2. what are the contents in the positions(154, 163 & 172 ) on the header and trailer record?

3. where does the money field start on the trailer record as you said it is at a different position

4. what is the recfm,lrecl of the files?

5. Please post a sample of input records and the desired output

Btw I don't think you can sum on 9 bytes on packed decimal field.Sort can handle a PD length up to 8 (15 digits and a sign), so if the length is really 9 (17 digits and a sign), that's a problem, unless the first byte of the PD field is always X'00'.

Let me know these details and I will post the solution.

Kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Wed Oct 29, 2003 12:26 pm    Post subject: Reply with quote

Kolusu said
Quote:
I don't think you can sum on 9 bytes on packed decimal field. Sort can handle a PD length up to 8 (15 digits and a sign)


Depends on which "Sort" you mean - DFSORT can handle a PD length up to 16 bytes for SUM.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Disco_Stu
Beginner


Joined: 04 Feb 2003
Posts: 19
Topics: 7

PostPosted: Wed Oct 29, 2003 12:31 pm    Post subject: Reply with quote

<b>I really don't see the use of an exit in here unless you are doing some complex programming logic. </b>

I need to update the trailer record with the new record count. if records will end up being combined, the count will decrease. I also want to merge all the trailer records into one. I dont want the final output to have more than one header or trailer record. Maybe there is an easier way to do this...


<b>1. How do you distinguish a header record, a detail record and the trailer record? </b>

header - 'CC' in pos 1-2
detail - 'DL' in pos 1-2
trailer - 'XX' in pos 1-2

<b>2. what are the contents in the positions(154, 163 & 172 ) on the header and trailer record? </b>

header - spaces

trailer - various money buckets, but the positions arent the same as the detail reocrd, so they dont match up with the detail layout, hence soc7


<b>3. where does the money field start on the trailer record as you said it is at a different position</b>

start at positions 162,171,180; each defined as PIC S9(15)V99 comp-3


<b>4. what is the recfm,lrecl of the files?</b>

293 bytes, fixed block

<b>5. Please post a sample of input records and the desired output </b>

I'll use an example here. i figured real data would be too big. I have 2 input datasets here. each has a header (CC) record, a few detail (DL) records, and a trailer (XX) record.
The header record is all spaces.
The deail reocord is made up of:
Positions 3-5 are my key.
At the end, 13-15 is a money field.
Trailer record has a record count in positions 3-5 and a total money field in positions 10-12.

Dataset 1

CC
DLAAABBBBBBB001
DLAAABBBBBBB000
DLAAABBBBBBB003
XX003AAAA004AAA

Dataset 2
CC
DLAAABBBBBBB002
DLAAABBBBBBB002
DLCCCBBBBBBB002
XX003AAAA006AAA

Output
CC
DLAAABBBBBBB008
DLCCCBBBBBBB002
XX002AAAA010AAA


All recs with key 'AAA' are combined. New trailer record with updated record count and money amt is created.

<b>Btw I don't think you can sum on 9 bytes on packed decimal field.Sort can handle a PD length up to 8 (15 digits and a sign), so if the length is really 9 (17 digits and a sign), that's a problem, unless the first byte of the PD field is always X'00'. </b>

See above for field definition. It's been working so far. but since the field is so big, the first byte is probably always X'00', like you said. I dont really have any freedom to change the field size.

<b>Depends on which "Sort" you mean - DFSORT can handle a PD length up to 16 bytes for SUM.</B>

I am using SyncSort
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Wed Oct 29, 2003 2:16 pm    Post subject: Reply with quote

Disco_stu,

The following JCL will give you the desired results. The only thing I was not able to do is the updation of the trailer record count.A brief explanation of the JOb.

I am using the change parm to change the contents of fields for header and trailer records.For every header & trailer record we are going to change the bytes 154, 163, 172 to have zero in packed decimal format.

I am also adding another 3 fields for which I am using the change operator to pad zeroes for every header & detail record. But for the trailer record I copy the contents from the bytes 162,171,180.

Now I summing the contents in 154, 163, 172, as well as the contents which we added i.e bytes 294,303,312

Now all the detailed records will be summed in the bytes 154 thru 180 and the trailer records will be summed in the bytes 294 thru 320.

Now using outrec fields I am adding the summed value in 155 bytes with the value in 295 bytes. similarly for the values in 164 bytes with the value in 304 .similary for the values in 173 with the value in 313. I am adding on 8 bytes as Sort can handle a PD length up to 8 and I padded a X'00' in front to the fields.

Now using change parm on outfil outrec I am padding spaces for the header record.

I haven't tested this code as I don't have the time to set up the input data.But you get the idea as to how it is being done.


Code:
   
//STEP0100 EXEC PGM=SORT 
//SYSOUT   DD SYSOUT=*   
//SORTIN   DD DISP=SHR,DSN=YOUR INPUT DSN1         
//         DD DISP=SHR,DSN=YOUR INPUT DSN2         
//         DD DISP=SHR,DSN=YOUR INPUT DSN3         
//         DD DISP=SHR,DSN=YOUR INPUT DSN4         
//         DD DISP=SHR,DSN=YOUR INPUT DSN5
//*
//SORTOUT  DD DSN=YOUR OUTPUT DSN,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD *
  SORT FIELDS=(1,63,CH,A)
  INREC FIELDS=(1,153,
               154:1,2,CHANGE=(9,C'CC',X'00000000000000000C',
                                 C'XX',X'00000000000000000C'),
                                 NOMATCH=(154,9),
               163:1,2,CHANGE=(9,C'CC',X'00000000000000000C',
                                 C'XX',X'00000000000000000C'),
                                 NOMATCH=(163,9),
               172:1,2,CHANGE=(9,C'CC',X'00000000000000000C',
                                 C'XX',X'00000000000000000C'),
                                 NOMATCH=(172,9),
               181,113,
               294:1,2,CHANGE=(9,C'CC',X'00000000000000000C',
                                 C'DL',X'00000000000000000C'),
                                 NOMATCH=(162,9),
               303:1,2,CHANGE=(9,C'CC',X'00000000000000000C',
                                 C'DL',X'00000000000000000C'),
                                 NOMATCH=(171,9),
               312:1,2,CHANGE=(9,C'CC',X'00000000000000000C',
                                 C'DL',X'00000000000000000C'),
                                 NOMATCH=(180,9))
  SUM FIELDS=(154,9,
             163,9,
             172,9,
             294,9,
             303,9,
             312,9),FORMAT=PD
  OUTREC FIELDS=(1,153,
                X'00',
                (155,8,PD,ADD,295,8,PD),LENGTH=8,
                X'00',
                (164,8,PD,ADD,304,8,PD),LENGTH=8,
                X'00',
                (173,8,PD,ADD,313,8,PD),LENGTH=8,
                181,113)
  OUTFIL OUTREC=(1,153,
                154:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(154,9)
                163:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(163,9)
                172:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(172,9)
                181,113)   




Hope this helps...

cheers

kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Wed Oct 29, 2003 2:22 pm    Post subject: Reply with quote

Frank,

Thanks for pointing that for summation DFSORT can handle upto 16 bytes of PD field.It is the same for syncsort also.But the max length on inrec/outrec fields is 8 for PD for syncsort.Does DFSORT allow more than 8 bytes for PD field on outrec?? I get a numeric field error if try to convert a zoned decimal to PD field with length of 9 bytes.

Kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Disco_Stu
Beginner


Joined: 04 Feb 2003
Posts: 19
Topics: 7

PostPosted: Wed Oct 29, 2003 3:15 pm    Post subject: Reply with quote

Thanks. There's alot of stuff in there I've never seen before. I'm gonna have to go through it and figure out the details. But from your description i get the basic idea.

Unfortunately, the count definitely needs to get updated. Otherwise i'll get out of balance errors down the line.

But you definitly exposed me to some new synsort code and some new ideas, and i think i'm going to be able to apply some of the things i see here to other processes, so thank you for your time.


Last edited by Disco_Stu on Wed Oct 29, 2003 3:27 pm; edited 1 time in total
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Wed Oct 29, 2003 3:18 pm    Post subject: Reply with quote

Kolusu said
Quote:
the max length on inrec/outrec fields is 8 for PD for syncsort. Does DFSORT allow more than 8 bytes for PD field on outrec??


No, that limit is 8 for DFSORT as well.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Disco_Stu
Beginner


Joined: 04 Feb 2003
Posts: 19
Topics: 7

PostPosted: Wed Oct 29, 2003 3:52 pm    Post subject: Reply with quote

One last question if you have the time...

whats the difference here between the OUTREC control statement
Code:
OUTREC FIELDS=(1,153,
                X'00',
                (155,8,PD,ADD,295,8,PD),LENGTH=8,
                X'00',
                (164,8,PD,ADD,304,8,PD),LENGTH=8,
                X'00',
                (173,8,PD,ADD,313,8,PD),LENGTH=8,
                181,113)

and the OUTREC parameter of the OUTFIL control statment?
Code:
 OUTFIL OUTREC=(1,153,
                154:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(154,9)
                163:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(163,9)
                172:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(172,9)
                181,113) 


From what i know, they seem to be very similar. So I'm just curious as to why you did what you did. could those outrec's been combined? or does this essentially allow you to process an outgoing record twice?
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Wed Oct 29, 2003 4:35 pm    Post subject: Reply with quote

Disco_Stu,

Yes I am reformating the output twice. I need to process the output record twice as I want sum up the quantities and then also changes the value to spaces on the header record.

Basically this is what happens after summing of the records

Code:

TYPE      MONEY1      MONEY2       MONEY3        TEMP1       TEMP2      TEMP3     
(1-2)    (155-162)    (164-171)   (173 - 180)    (295-302)   (304-311)  (313-320)
CC          0            0            0             0          0           0
DL A       1000        2000         3000            0          0           0
DL B       1300        2200          300            0          0           0       
...
XX         0             0            0            4000       5000        6000


Now the first outrec fields will sum

Money1 with TEMP1
Money2 with TEMP2
Money3 with TEMP3

Now the outrec on the outfil statements takes these updated values and changes the zeroes on the header record to have spaces.Basically we are processing the output record twice.

To get the updated record you can use the trailer feature on the outfil and it will give you the new count of records which is inclusive of the header and trailer record.

Hope this explanation is clear

Kolusu


Last edited by kolusu on Wed Oct 29, 2003 5:49 pm; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Thu Oct 30, 2003 9:12 am    Post subject: Reply with quote

Disco_stu,

I need small clarification from you about trailer record. Where does the count of records fit in? i.e the position of the record count.

For that position what are the contents on the header and detail records.

You can have the updated record count also with some adjustments. Are you willing increase the LRECL of the output dataset??

Kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Disco_Stu
Beginner


Joined: 04 Feb 2003
Posts: 19
Topics: 7

PostPosted: Thu Oct 30, 2003 9:29 am    Post subject: Reply with quote

This is what the trailer record looks like.
Code:

  1   02  STATDB-LS-CTL-REC REDEFINES STATDB-LS-DETAIL-REC.     
  1     05  STATDB-LS-CTL-REC-TYPE              PIC X(2).       
  3     05  STATDB-LS-CTL-DATE.                                 
  3         15  STATDB-LS-CTL-YYYY              PIC X(4).       
  7         15  STATDB-LS-CTL-MM                PIC X(2).       
  9     05  STATDB-LS-CTL-SRC-SYS-CD            PIC X(8).       
 17     05  STATDB-LS-CTL-SEQ-NBR               PIC X(3).       
 20     05  STATDB-LS-CTL-LEDG-CO-CD            PIC X(2).       
 22     05  STATDB-LS-CTL-SUBGRP-CD             PIC X(1).       
 23     05  STATDB-LS-CTL-REC-CT                PIC 9(7) COMP-3.
 27     05  STATDB-LS-CTL-AMOUNTS OCCURS 6 TIMES COMP-3.       
 27         15  STATDB-LS-CTL-FILLER1           PIC S9(15)V99. 
 81     05  STATDB-LS-CTL-RSV-AMOUNTS      COMP-3.             
 81         15  STATDB-LS-CTL-CASE-ALAE-RSV     PIC S9(15)V99. 
 90         15  STATDB-LS-CTL-CASE-LOSS-RSV     PIC S9(15)V99. 
 99     05  STATDB-LS-CTL-AMOUNTS OCCURS 7 TIMES COMP-3.       
 99         15  STATDB-LS-CTL-FILLER2           PIC S9(15)V99. 
162     05  STATDB-LS-CTL-PD-LOSS-AMOUNTS      COMP-3.         


The header record
Code:

 1 01  STATDB-LS-REC.                                         
 1   02  STATDB-LS-CYC-REC.                                   
 1     05  STATDB-LS-CYC-REC-TYPE                  PIC X(2).   
 3     05  STATDB-LS-CYC-DATE.                                 
 3         10  STATDB-LS-CYC-YYYY                  PIC X(4).   
 7         10  STATDB-LS-CYC-MM                    PIC X(2).   
 9     05  STATDB-LS-CYC-SRC-SYS-CD                PIC X(8).   
17     05  STATDB-LS-CYC-SEQ-NBR                   PIC X(3).   
20     05  STATDB-LS-CYC-FILLER1                   PIC X(274).



Adjusting the LRECL isnt really an option. The count is in positin 23 for 4 bytes. That is spaces on the header record, and overlaps a couple alphanumeric fields on the detail record.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Thu Oct 30, 2003 10:21 am    Post subject: Reply with quote

Disco_stu,


If the alphanumeric fileds in position 23 for 4 bytes on the header and detail records are not that important then you can add a seqnum to outrec fields and in the end subtract 2 from it so that you will get the record count.

Code:

  OUTREC FIELDS=(1,22,
                SEQNUM,4,ZD,
                27,127,
                X'00',
                (155,8,PD,ADD,295,8,PD),LENGTH=8,
                X'00',
                (164,8,PD,ADD,304,8,PD),LENGTH=8,
                X'00',
                (173,8,PD,ADD,313,8,PD),LENGTH=8,
                181,113)
  OUTFIL OUTREC=(1,22,
                +2,SUB,23,4,ZD,EDIT=(TTTT),
                27,127,   
                154:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(154,9)
                163:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(163,9)
                172:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(172,9)
                181,113)   



Just incase if you are willing to increase the LRECL, then you add the seqnum at the end so that you will keep all the values(pos 23 - 26) of header and detail records intact.

Code:


  OUTREC FIELDS=(1,153,
                X'00',
                (155,8,PD,ADD,295,8,PD),LENGTH=8,
                X'00',
                (164,8,PD,ADD,304,8,PD),LENGTH=8,
                X'00',
                (173,8,PD,ADD,313,8,PD),LENGTH=8,
                181,113,
                SEQNUM,4,ZD)
  OUTFIL OUTREC=(1,153,
                154:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(154,9)
                163:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(163,9)
                172:1,2,CHANGE=(9,C'CC',C'        '),
                                  NOMATCH=(172,9)
                181,113,
                +2,SUB,294,4,ZD,EDIT=(TTTT))


With the above code you will have the new record count at the end of the record in pos(294-297)

Hope this helps...

cheers

kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group