MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Sorting and merging 2 files AND indicating source file

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
misi01
Advanced


Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden

PostPosted: Tue Dec 03, 2013 10:43 am    Post subject: Sorting and merging 2 files AND indicating source file Reply with quote

First of all, I'll say straight off that I (basically) never write/create icetool/dfsort JCL other than the most trivial there is.

If (almost) found what I thought might be the solution here (http://www.mvsforums.com/helpboards/viewtopic.php?t=11573&highlight=insert) but it didn't quite seem what I was after.

I would like to merge 2 files into one, sorting them at the same time based on a couple of fields. At the same time I would also like to be able to insert a field indicating from which file the source came. In its simplest form, I have 2 input files with the last qualifier as A1234567 and B1234567. I would like to merge them and at the same time insert the values A1234567 or B1234567 into the output file depending on where the input record came from.
_________________
Michael
Back to top
View user's profile Send private message Send e-mail
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Tue Dec 03, 2013 11:10 am    Post subject: Reply with quote

misi01,

It is quite easy to do it, however it would have been great if you had posted a sample of input and desired output.

When you say merge do you mean merging the contents of 2 files into a single record in case of match or just plain merge of record one after another based on the key?

Please post the LRECL and RECFM of the input/Output datasets along with the positions and the format of the Key fields to be sorted.
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
misi01
Advanced


Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden

PostPosted: Wed Dec 04, 2013 1:07 am    Post subject: Sorry, Kolusu, unusually stupid of me Reply with quote

I expressed myself incorrectly inasmuch as I know how to express the really basic sort parms, it was the insert of an extra field based on the selected record that I don't know.

Both input files are VB, LRECL 165. The output file will obviously be VB 173 (8 bytes for the name of the source file), but if it's easier to make it FB, 173, that's not a problem.

File 1 (A1234567)
Code:

0This is date record 1
1FRED1111Data after identifier
1FRED0011Data after identifier
1FRED2222Data after identifier


File 2 (B1234567)
Code:

0This is date record 2
1FRED1111Data after identifier
1FRED3333Data after identifier
1FRED2222Data after identifier


Sorted records should be something like
Code:

0This is date record 1        A1234567
0This is date record 2        B1234567
1FRED0011Data after identifierA1234567
1FRED1111Data after identifierA1234567
1FRED1111Data after identifierB1234567
1FRED2222Data after identifierA1234567
1FRED2222Data after identifierB1234567
1FRED3333Data after identifierB1234567


The records are sorted on column 1 first , then on the columns between FRED and Data after identifier (cols 6-9)

NB The order in which duplicate records from both files 1 & 2 end up in the output file doesn't matter, just so long as I can see from which input file they came.
_________________
Michael
Back to top
View user's profile Send private message Send e-mail
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Wed Dec 04, 2013 12:10 pm    Post subject: Reply with quote

misi01,

Use the following DFSORT JCL which will give you the desired results

Code:

//**********************************************************************
//* CREATE A DUMMY HEADER TO IDENTIFY THE FILE RECORDS                 *
//**********************************************************************
//STEP0100 EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD *                                                         
//SORTOUT  DD DSN=&&HDR,DISP=(,PASS),SPACE=(TRK,(1,0),RLSE)             
//SYSIN    DD *                                                         
  OPTION COPY                                                           
  INREC BUILD=(161X)                                                   
  OUTFIL REMOVECC,NODETAIL,FTOV,HEADER1=('$$$')                         
//*                                                                     
//**********************************************************************
//* CONCATENATE THE HEADER FILE WITH VB FILES AND USE WHEN=GROUP TO    *
//* IDENTIFY AND TAG THE RECORDS WITH A CONSTANT VALUE                 *
//**********************************************************************
//STEP0200 EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD DSN=&&HDR,DISP=SHR,VOL=REF=*.STEP0100.SORTOUT             
//         DD DSN=Your VB Input File 1,DISP=SHR
//         DD DSN=&&HDR,DISP=SHR,VOL=REF=*.STEP0100.SORTOUT             
//         DD DSN=Your VB Input File 2,DISP=SHR
//SORTOUT  DD SYSOUT=*                                                 
//SYSIN    DD *                                                         
  INREC IFOUTLEN=173,                                                   
  IFTHEN=(WHEN=INIT,OVERLAY=(166:C'A1234567')),                         
  IFTHEN=(WHEN=GROUP,BEGIN=(5,3,CH,EQ,C'$$$'),PUSH=(174:ID=1)),         
  IFTHEN=(WHEN=(174,1,ZD,EQ,2),OVERLAY=(166:C'B'))                     
                                                                       
  SORT FIELDS=(05,1,CH,A,                                               
               10,4,CH,A)                                               
                                                                       
  OUTFIL OMIT=(5,3,CH,EQ,C'$$$')                                       
//*

_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
misi01
Advanced


Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden

PostPosted: Wed Dec 04, 2013 12:30 pm    Post subject: Reply with quote

Thanks a lot. I'll try that tomorrow at work (and I'd NEVER have figured that out myself)
_________________
Michael
Back to top
View user's profile Send private message Send e-mail
misi01
Advanced


Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden

PostPosted: Thu Dec 05, 2013 5:59 am    Post subject: This is what I got from doing a JJ on the JCL Reply with quote

(by JJ, I mean, in our system, check that the JCL is valid).

2 comments.

1) I changed the second sort field to the relevant one for the files.
2) I'll try submitting the JCL just to see if it works despite the error messages

Code:

  INREC IFOUTLEN=173,                                         
  IFTHEN=(WHEN=INIT,OVERLAY=(166:C'A1234567')),               
  IFTHEN=(WHEN=GROUP,BEGIN=(5,3,CH,EQ,C'$$$'),PUSH=(174:ID=1)),
  IFTHEN=(WHEN=(174,1,ZD,EQ,2),OVERLAY=(166:C'B'))             
E2 - DSS10078E - 'INIT' IS MUTUALLY EXCLUSIVE WITH 'BEGIN'.   
E3 - DSS10078E - 'INIT' IS MUTUALLY EXCLUSIVE WITH 'PUSH'.     
E4 - DSS10078E - 'GROUP' IS MUTUALLY EXCLUSIVE WITH 'OVERLAY'.
                                                               
  SORT FIELDS=(05,01,CH,A,                                     
               57,11,CH,A)                                     

_________________
Michael
Back to top
View user's profile Send private message Send e-mail
misi01
Advanced


Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden

PostPosted: Thu Dec 05, 2013 6:09 am    Post subject: Reply with quote

An append to my last one. JJ might have complained about the JCL, but it worked like a charm !!!!

Many thanks Kolusu (I showed it to a colleague who was suitably impressed)
_________________
Michael
Back to top
View user's profile Send private message Send e-mail
William Collins
Supermod


Joined: 03 Jun 2012
Posts: 437
Topics: 0

PostPosted: Thu Dec 05, 2013 6:48 am    Post subject: Reply with quote

Yes, ignore any JCL-checker messages about Sort Control Cards, the JCL-checkers seem to have no clue.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12378
Topics: 75
Location: San Jose

PostPosted: Thu Dec 05, 2013 11:14 am    Post subject: Re: This is what I got from doing a JJ on the JCL Reply with quote

misi01 wrote:
(by JJ, I mean, in our system, check that the JCL is valid).
E2 - DSS10078E - 'INIT' IS MUTUALLY EXCLUSIVE WITH 'BEGIN'.
E3 - DSS10078E - 'INIT' IS MUTUALLY EXCLUSIVE WITH 'PUSH'.
E4 - DSS10078E - 'GROUP' IS MUTUALLY EXCLUSIVE WITH 'OVERLAY'.


misi01,

You can ignore the JCL checker messages , as they never keep up with the new features of DFSORT. Earlier the checker would not recognize the new keywords but it seems to have set up its own rules about the keywords. I wonder where they got the idea of "MUTUALLY EXCLUSIVE" parms. The documentation is pretty clear about WHEN=INIT and WHEN=GROUP . *Sigh* Wish they could contact us before defining their own rules.
misi01 wrote:

An append to my last one. JJ might have complained about the JCL, but it worked like a charm !!!!

Many thanks Kolusu (I showed it to a colleague who was suitably impressed)


Glad the proposed solution worked for you. This solution is just one of the many cool things that you can do with DFSORT. Give it a test and see what it can do. Smile
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group