MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Write only first n messages in OUTFILs with Syncsort

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
mcclr
Beginner


Joined: 10 Jan 2006
Posts: 11
Topics: 4

PostPosted: Mon Jul 09, 2007 1:03 pm    Post subject: Write only first n messages in OUTFILs with Syncsort Reply with quote

We are trying to read an input file and split it into 3 files depending on values in columns 1 to 4. We only want the first 700000 records in each of the 3 OUTFILs. We believe that we could do it in DFSORT by using IFTHEN OUTREC logic and adding a sequence number then comparing the OUTFIL include to also check that the sequence number is less than 700001. How can we do this with Syncsort which does not support he IFTHEN logic?
Back to top
View user's profile Send private message
amargulies
Beginner


Joined: 10 Jan 2007
Posts: 123
Topics: 0

PostPosted: Mon Jul 09, 2007 1:51 pm    Post subject: Reply with quote

mcclr,

SyncSort for z/OS 1.2 does support IFTHEN but that is probably not what you need. If you are looking to divide your output into 3 files based on the contents of particular field but you only want 7000 records in each output, we can do that much easier. Try the following code:

Code:
  SORT FIELDS=(.....)
  OUTFIL FNAMES=FILE1,ENDREC=7000,
     INCLUDE=(1,4,.....)
  OUTFIL FNAMES=FILE2,ENDREC=7000,
    INCLUDE=(1,4,.....)
  OUTFIL FNAMES=FILE3,ENDREC=7000,
    INCLUDE=(1,4,.....)


There are many other ways to break up the data being written to different OUTFILs. If this doesn't work for you let me know exactly what is wrong and we can work it out.
_________________
Alissa Margulies
SyncSort Mainframe Product Services
201-930-8260
zos_tech@syncsort.com
Back to top
View user's profile Send private message Send e-mail
CICS Guy
Intermediate


Joined: 30 Apr 2007
Posts: 292
Topics: 3

PostPosted: Mon Jul 09, 2007 1:54 pm    Post subject: Reply with quote

What level of Syncsort are you at, above 1.2 it does support IFTHEN logic.
Anyway, you should be able to OUTFIL FILES=nn,INCLUDE=,ENDREC= without the IFTHEN logic.
Back to top
View user's profile Send private message
mcclr
Beginner


Joined: 10 Jan 2006
Posts: 11
Topics: 4

PostPosted: Mon Jul 09, 2007 3:04 pm    Post subject: Reply with quote

Thanks for the update, Alisssa. I tried your suggestion, but only the first OUTFIL contained any records. My understanding of the ENDREC parm is that it is specifying how many input records to process before ending rather than how many output records to process. If my interpretation is not correct, I can paste in my small test case so that you can check my input statements.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12375
Topics: 75
Location: San Jose

PostPosted: Mon Jul 09, 2007 6:03 pm    Post subject: Reply with quote

Alissa/Cics Guy,

ENDREC is processed before INCLUDE/OMIT conditions on OUTFIL. So only the first file will have the output and rest of the output files will be empty which is confirmed by OP.

mcclr,

I am not sure what version of syncsort you are running, but if you are interested I can show you a way to get the desired results but would involve 2 passes of data. If that is ok let me know the LRECL & RECFM of input & output files.

or if your shop has fileaid then you can use the following JCL to get the desired results

Code:

//STEP0100 EXEC PGM=FILEAID
//SYSPRINT DD SYSOUT=*     
//DD01     DD DSN=YOUR INPUT FILE,
//            DISP=SHR
//OUTPUT1  DD SYSOUT=*                       
//OUTPUT2  DD SYSOUT=*                       
//OUTPUT3  DD SYSOUT=*                       
//SYSIN    DD *                               
$$DD01 USER IF=(1,EQ,C'AAAA'),WRITE=OUTPUT1,OUT=70000
$$DD01 USER IF=(1,EQ,C'BBBB'),WRITE=OUTPUT2,OUT=70000
$$DD01 USER IF=(1,EQ,C'CCCC'),WRITE=OUTPUT3,OUT=70000
/*



Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mcclr
Beginner


Joined: 10 Jan 2006
Posts: 11
Topics: 4

PostPosted: Tue Jul 10, 2007 10:42 am    Post subject: Reply with quote

Thanks kolusu. We would prefer doing this in one pass, but 2 is better than the 4 we may end up with. The lrecl is 243 and the recfm is fb.

Alissa,
I would also be interested in any other ways that you may have to accomplish this.

Alissa and Frank,
Would it be inappropriate to suggest an enhancement request for both products to add a STOPAFT option to the OUTFIL statement? That would seem to be a fairly simple solution.

Larry
Back to top
View user's profile Send private message
amargulies
Beginner


Joined: 10 Jan 2007
Posts: 123
Topics: 0

PostPosted: Tue Jul 10, 2007 12:23 pm    Post subject: Reply with quote

Larry,

I had problems with the web site yesterday. I did not mean to post the message that was posted. I was trying to PREVIEW the message while I tested the solution and somehow the message got posted multiple times. To make matters worse for the solution to work there has to be too many restrictions on the input data that makes it weak.

A better solution is the following:

[code:1:40602e44dd]
//SYSIN DD *
SORT FIELDS=(1,4,BI,A)
OUTREC FIELDS=(1,80,SEQNUM,12,ZD,RESTART=(1,4))
OUTFIL FNAMES=OUT1,
INCLUDE=(1,4,BI,EQ,X
_________________
Alissa Margulies
SyncSort Mainframe Product Services
201-930-8260
zos_tech@syncsort.com
Back to top
View user's profile Send private message Send e-mail
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12375
Topics: 75
Location: San Jose

PostPosted: Tue Jul 10, 2007 1:07 pm    Post subject: Reply with quote

mcclr,

Here is the JCL which will give you the desired results.Basically we split the input files into 3 different files and while doing that we add a seqnum to each output file and in the next step we use that seqnum to pick records which are less than or equal to desired count.


Code:

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*   
//SORTIN   DD DSN=YOUR INPUT FILE,
//            DISP=SHR
//OUT1     DD DSN=&T1,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//OUT2     DD DSN=&T2,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//OUT3     DD DSN=&T3,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD *                     
  SORT FIELDS=COPY                   

  OUTFIL FNAMES=OUT1,               
  INCLUDE=(1,4,CH,EQ,C'AAAA'),         
  OUTREC=(01,243,SEQNUM,8,ZD)
       
  OUTFIL FNAMES=OUT2,               
  INCLUDE=(1,4,CH,EQ,C'BBBB'),         
  OUTREC=(01,243,SEQNUM,8,ZD)       

  OUTFIL FNAMES=OUT3,               
  INCLUDE=(1,4,CH,EQ,C'CCCC'),         
  OUTREC=(01,243,SEQNUM,8,ZD)       
/*
//STEP0200 EXEC PGM=SORT                       
//SYSOUT   DD SYSOUT=*                         
//SORTIN   DD DSN=&T1,DISP=SHR                 
//         DD DSN=&T2,DISP=SHR                 
//         DD DSN=&T3,DISP=SHR                 
//OUT1     DD DSN=YOUR OUTPUT FILE1,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
//*
//OUT2     DD DSN=YOUR OUTPUT FILE2,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
//*
//OUT3     DD DSN=YOUR OUTPUT FILE3,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
//*
//SYSIN    DD *                     
  INCLUDE COND=(244,8,ZD,LE,70000)                 
  INREC FIELDS=(01,243)                         
  SORT FIELDS=COPY                             

  OUTFIL FNAMES=OUT1,INCLUDE=(1,4,CH,EQ,C'AAAA')
  OUTFIL FNAMES=OUT2,INCLUDE=(1,4,CH,EQ,C'BBBB')
  OUTFIL FNAMES=OUT3,INCLUDE=(1,4,CH,EQ,C'CCCC')
/*



Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12375
Topics: 75
Location: San Jose

PostPosted: Tue Jul 10, 2007 1:17 pm    Post subject: Reply with quote

Alissa,

I think OP is running a version of syncsort which is less than syncsort z/os 1.2 version and most of the features like IFTHEN & RESTART parameters are only supported in versions 1.2 and higher. So he/she needs a solution which would proably work even on on syncsort's older versions.

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Tue Jul 10, 2007 1:24 pm    Post subject: Reply with quote

Quote:
Would it be inappropriate to suggest an enhancement request for both products to add a STOPAFT option to the OUTFIL statement? That would seem to be a fairly simple solution.


Since I "invented" ENDREC, let me explain the difference between what STOPAFT does and what ENDREC does.

STOPAFT=n stops after n input records are accepted. So let's say we have STOPAFT=5, an INCLUDE COND=(...), 20 input records and records 3-7 meet the INCLUDE criteria. We will stop processing at record 7 with records 3-7 in the output file.

ENDREC=n stops at record n regardless of how many records are accepted. So let's say we have OUTFIL with ENDREC=5, an INCLUDE=(...), 20 input records and records 3-7 meet the INCLUDE criteria. We will stop processing at record 5 with records 3-5 in the output file.

What's missing is STOPAFT=n for OUTFIL which would stop after n records are accepted. So let's say we have OUTFIL with STOPAFT=5, an INCLUDE=(...), 20 input records and records 3-7 meet the INCLUDE criteria. We will stop processing at record 7 with records 3-7 in the output file.

I will add STOPAFT for OUTFIL to the list of candidates for future enhancements to DFSORT.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mcclr
Beginner


Joined: 10 Jan 2006
Posts: 11
Topics: 4

PostPosted: Tue Jul 10, 2007 5:31 pm    Post subject: Reply with quote

Thanks for the help.

Kolusu's solution of two steps is superior to our 4 step solution.

Thanks to Frank for the enhancement request for a STOPAFT option on OUTFIL and for the IFTHEN WHEN SEQNUM example that we found in another post.

Thanks to Alissa also. It appears that we will need to get release 1.2 of Syncsort and use Frank's IFTHEN WHEN SEQNUM logic because our data is not numeric but rather character and we want one file to contain data < 'CN ' the second file < 'OPTX' and the third file to contain the remainder - unless you have another idea that will work for that scenario.

Thanks, Larry
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Thu Oct 28, 2010 3:19 pm    Post subject: Reply with quote

Quote:
I will add STOPAFT for OUTFIL to the list of candidates for future enhancements to DFSORT.


With z/OS DFSORT V1R10 PTF UK90025 or z/OS DFSORT V1R12 PTF UK90026 (Oct,2010), DFSORT now has OUTFIL ACCEPT=n to perform this function:

Code:

  SORT FIELDS=(.....)
  OUTFIL FNAMES=FILE1,ACCEPT=7000,
     INCLUDE=(1,4,.....)
  OUTFIL FNAMES=FILE2,ACCEPT=7000,
    INCLUDE=(1,4,.....)
  OUTFIL FNAMES=FILE3,ACCEPT=7000,
    INCLUDE=(1,4,.....)


For complete details on the new functions for DFSORT and DFSORT's ICETOOL available with the Oct, 2010 PTF, see:

http://www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000242
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group