Posted: Wed Apr 20, 2005 6:10 am Post subject: E35 performance tuning
Hello,
I need to improve efficiency/performance for a sort that uses the E35 exit routine. I read in the manuals that by coding the OPTION AVGRLEN
can improve efficiency. I am giving that a try. One more way is to get away with the E35 routine and instead use either OMIT,INCLUDE etc to achieve the same functionality. The exit routine has the following logic in cobol and I want to achieve the same in Syncsort :
Say Input-rec is In-rec and output-rec is Out-rec
Code:
Move In-rec to Out-rec.
Move 'Any Constant' to Out-rec(40:5).
If Out-rec(80:3) = 'abc'
Move Out-rec(60:2) to Out-rec(35:2), X1
If Out-rec(65:2) = 'ab' 'bc' 'cd'
Move 'y' to Out-rec(37:1), X2
else
Move 'n' to Out-rec(37:1), X2
end-if
else
Move X1 to Out-rec(35:2)
Move X2 to Out-rec(37:1)
end-if.
All input records should be there on the output file.
If there are any other ways in which performance can be improved for such routines, please suggest. Any help is appreciated.
Thankyou very much for your prompt reply.Sorry I forgot to mention my input file attributes. It is VB with LRECL as 1100. Could you please explain the control statements in a little detail ? I could understand somepart of it, but still not very clear with the whole thing and I think VB shouldn't be a problem as I will try increasing the bytes by 4.
My outfile also has LRECL 1100 and RECFM as VB. Please let me know how to code the same for a VB file. I tried by increasing the bytes by 4 , but the error I am getting is as below:
WER108I SORTIN : RECFM=VB ; LRECL= 1100; BLKSIZE= 27998
WER257I INREC RECORD LENGTH = 1108
WER238I POTENTIALLY INEFFICIENT USE OF INREC
WER238I POTENTIALLY INEFFICIENT USE OF INREC
EXPLANATION: The INREC control has been used to lengthen the
input record length. This can reduce SyncSort performance because a
larger volume of data is being processed than if the OUTREC control
statement were used to perform the same function. Typically,
increasing the record length
with INREC is only useful when expanding SUM fields with leading
zeros to prevent an overflow condition during SUM.
ACTION: Revise the application so that addition of data is performed in an
OUTREC statement. Be sure to adjust the FIELDS of the SORT,
MERGE or SUM control statements if necessary.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Fri Apr 22, 2005 5:41 am Post subject:
HBo,
A Vb file ! yikes! It is too early in the moring here and I just not in a mood to explain the control cards . However you can try the following control cards(not tested) for vb files. I will post the explanation later in the afternoon.
Code:
INREC FIELDS=(01,04, $ RDW
80,03, $ COPY 3 BYTES FROM 80
65,2, $ COPY 2 BYTES FROM 65
05) $ ACTUAL DATA
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Fri Apr 22, 2005 1:07 pm Post subject:
HBo,
Here is an explanation of the control cards
Code:
INREC FIELDS=(01,04, $ RDW
80,03, $ COPY 3 BYTES FROM 80
65,2, $ COPY 2 BYTES FROM 65
05) $ ACTUAL DATA
Since your input file is a variable block file, Using INREC FIELDS we add the validation bytes('abc' at 80 and 'ab' 'bc' 'cd' at position 65) at the beginning of every record.
A typical variable block file looks as follows
Code:
1-4 | 5- 1096
----------------
RDW1 | YOUR DATA
RDW2 | YOUR DATA
..
So with the inrec statement this how your data looks like
Code:
1-4 | 5-10 | 10- 1101
------------------------
RDW1 | abcab | YOUR DATA
RDW2 | abcbc | YOUR DATA
...
Note that your LRECL is increased by 5 bytes now. For variable block files, you can just specify the starting position from where you want to copy . You don't have to specify the length. so I just specified 5 at the end so that it will copy the entire record even if you have short records.
Sort fields are self explanatory. However I am confused as to why you have 2 statements for sorting fields at 665.
Code:
655,001,CH,A, <<<- same position for 1 byte
655,6,CH,A <<<- same position for 6 bytes
we are copying the records as is while making changes to a couple of fields .Now using CHANGE command on outrec fields we first change the byte 9 to a 'y' if it has abc in the position 5 for 3 bytes ( remember that you copied the contents of 80th byte to byte 5) . Since we add 5 extra bytes at the beginning of every record , now the actual change postion(35) is shifted to 40 and 37 is shifted 42.
we use another CHANGE command to change contents at 40(original 35) validating the bytes 85(original 80) .
Similary we use another change command to change the contents at 42(original 37) valiadating the bytes at 5. In your pgm you are looking for ab bc cd only if the 80th byte is abc. so I am concatenating them together as abcab , abcbc, abccd as a single string to match the if condition in your pgm.
Also you wanted to add 'any constant' at byte 40. so we add constant at 45 (due to 5 bytes at the beginning of the record)
And we just code the start position to copy the contents from byte 50 till the end.
I have the following questions after looking at your reply.
1. As written earlier, we are increasing the length of our Input file so it is giving a warning POTENTIALLY INEFFICIENT USE OF INREC
and this says that it degrades the performance of sort. When we are actually trying to improve performance, Isn't this kind of warning not good for us ?
Does it mean that you are changing the 8th byte to a 'Y' or the 9th. and what about the figure 35. And does 05,03, mean you are taking 3 bytes starting from 5 as it is. And when you say 08:051,3 ,should this be 05 and not 051 ?
Does this line 35:40,2,CHANGE=(2,C' ',C' '),NOMATCH=(65,2), say that 35th position have 40,2 from code in OUTREC FIELDS if space, else have 65,2 from INREC FILEDS.
Does this line 37:8,1,CHANGE=(1,C' ',C' '),NOMATCH=(42,1), say that 37th position have 8,1 from code in INREC FIELDS if space else have 42,1 in OUTREC FIELDS.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Mon Apr 25, 2005 8:55 am Post subject:
HB0,
Syncsort flags the use of INREC as potentially inefficient use of inrec because it treats as the larger lrecl will degrade the performance. But I guess adding 5 bytes is not going to degrade the performance. So why not run both versions and compare the results?
Quote:
Does it mean that you are changing the 8th byte to a 'Y' or the 9th. and what about the figure 35. And does 05,03, mean you are taking 3 bytes starting from 5 as it is. And when you say 08:051,3 ,should this be 05 and not 051 ?
yes we are changing the 8th byte to 'y' and it should be 5 instead of 51 . The 51 was a typo.
Quote:
Does this line 35:40,2,CHANGE=(2,C' ',C' '),NOMATCH=(65,2), say that 35th position have 40,2 from code in OUTREC FIELDS if space, else have 65,2 from INREC FILEDS.
We are changing the contents at 35 for 2 bytes by validating the contents at 40 for 2 bytes. If the contents at 40 are spaces then code the space , if not then put the contents from 65th byte at 35th byte.
Quote:
Does this line 37:8,1,CHANGE=(1,C' ',C' '),NOMATCH=(42,1), say that 37th position have 8,1 from code in INREC FIELDS if space else have 42,1 in OUTREC FIELDS.
if you code 37:8,1 that means you are working with fields at 37th byte , but you are validating that with fields at pos 8 for 1 byte. if pos 8 has a space then code a space at pos 37 , if it is not a space then put the contents at pos 42 in pos 37.
Will our intermediate output after this is like
1.RDW1ABCYINPUT35BYTESNOYCONSTINPUTACTUALDATA
OR
2.RDW1ABC INPUT35BYTES NCONSTINPUTACTUALDATA
This line 09,35, says that we are at the 9 th Byte and we are taking 35 bytes from input as we are not changing the input upto 35 bytes. Is that right ?If yes, then when we add 9 and 35 it is 44 right and we are starting at 40 after that, So isn't there an overlap ? I am getting the error as follows :
WER136A OUTREC HAS OVERLAPPING COLUMNS SPECIFIED
And my output file should also be VB with 1100 has LRECL. Please let me know if this holds good with the code we are following.
I have a similar requirement in two programs,except for change in positions. One is what I posted here and the other one is as follows. My thought was to get the basic idea from here and incorporate it in both.
My input and output are VB with LRECL as 1100 bytes.
The other code is as follows :
Code:
MOVE 'GSDS BILLS' TO OUT-REC(36:10).
IF OUT-REC(646:6) = '111000'
MOVE OUT-REC(673:2) TO OUT-REC(33:2)
NBR-OF-COPIES
IF OUT-REC(736:2)= 'US' OR 'DC' OR 'GS' OR 'IR' OR 'FC'
MOVE 'Y' TO OUT-REC(35:1)
G-IND
ELSE
MOVE 'N' TO OUT-REC(35:1)
G-IND
END-IF
ELSE
MOVE NBR-OF-COPIES TO OUT-REC(33:2)
MOVE G-IND TO OUT-REC(35:1)
END-IF.
I have coded the following as per your suggestions. I had to use SORT FIELDS=COPY as the SORT FIELDS and OUTREC FIELDS together were giving me an error.So I sorted the fields in a step prior to this and used COPY here.
As for the 2nd requirement your input file has short records. That is causing the job to abend. Take a look at the following message in your sysout.
Code:
WER244A INREC - SHORT RECORD
The explanation of the error is :
Code:
WER244A [ddname] {INREC,OUTREC} SHORT RECORD
EXPLANATION: The ddname will be SORTOUT, SORTOFxx,
SORTOFx or the ddname provided by an OUTFIL FNAMES parameter.
A variable-length record was too short to contain all the fields
specified on the control statement. Program HISTOGRM may be used to
determine the length of the shortest record in the input file.
Run the following job and check the LONGEST and SHORTEST record.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Quick Manuals
Register
Profile
Log in to check your private messages
Log in
