MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Problem with splicing

 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Thu Sep 23, 2004 9:32 am    Post subject: Problem with splicing Reply with quote

File A LRECL=80 RECFM=FB
Lookup file
Code:

1-9   10-16
123456 IDSX
567015 IDSY
345678 IDSX
906615 IDSY


File B LRECL=446 RECFM=FB
Looks something like this
Code:

123456 xxa .....
30000  abc .....
123456 def......
345678 fgh......
567015 nom......
345678 eti......
906615 lkg......

Code:

Output         156-6      366-1
123456 xxa              IDSX      I
30000  abc           
123456 def              IDSX            I
345678 fgh              IDSX            I
567015 nom              IDSY            J
345678 eti              IDSX            I
906615 lkg              IDSY            J


Here is what I coded to get this
Code:

//SSORT1  EXEC PGM=ICETOOL                                           
//TEMP1 DD DSN=&&TEMP1,UNIT=SYSDA,SPACE=(CYL,(100,100)),DISP=(,PASS)
//TEMP2 DD DSN=&&TEMP2,UNIT=SYSDA,SPACE=(CYL,(100,100)),DISP=(,PASS)
//CON DD DSN=*.TEMP2,VOL=REF=*.TEMP2,DISP=(OLD,PASS)                 
//    DD DSN=*.TEMP1,VOL=REF=*.TEMP1,DISP=(OLD,PASS)                 
//TOOLIN   DD  *                                                     
  SPLICE FROM(CON) TO(OUT1) ON(1,9,CH) WITH(156,6) -         
  WITH(366,1) WITHALL KEEPNODUPS                             
  SORT FROM(OUT1) TO(OUT2) USING(CTL3)                       
//OUT1 DD DSN=out1 dsname,
//             DISP=(,CATLG,CATLG),UNIT=DEVPOOL,             
//             DSORG=PS,RECFM=FB,LRECL=454,                 
//             SPACE=(CYL,(200,250),RLSE)                   
//OUT2 DD DSN=out2 dsname,       
//             DISP=(,CATLG,CATLG),UNIT=DEVPOOL,             
//             DSORG=PS,RECFM=FB,LRECL=446,                 
//             SPACE=(CYL,(200,250),RLSE)                   
//SYSOUT   DD  SYSOUT=*                                     
//IN1 DD DSN=in1 dsname,DISP=SHR
//IN2 DD DSN=in2 dsname,DISP=SHR
//CTL2CNTL DD *                                             
  OUTREC FIELDS=(1,446,447:447,8)               
/*                                             
//CTL1CNTL DD *                                 
  SORT FIELDS=(1,9,CH,A)                       
  OUTREC FIELDS=(1,9,10:146X,156:10,6,162:204X,
          366:10,6,CHANGE=(1,C'IDSX  ',C'I',   
                              C'IDSY',C'J'), 
                    NOMATCH=(C' '),367:80X,     
                    447:8X)                     
//CTL3CNTL DD *                                 
  SORT FIELDS=(447,8,ZD,A)                     
  OUTREC FIELDS=(1,446,447:447,8)
/*                                             
//TOOLMSG  DD  SYSOUT=*                         
//DFSMSG   DD  SYSOUT=* 


Can you tell me what is wrong with my code?? My output does not contain the fields i had wanted to splice.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Thu Sep 23, 2004 10:01 am    Post subject: Reply with quote

Mfjin,

I cannot relate your JOb to your output.You are missing a copy statement in the TOOLIN for formating the lookup(80 byte) file to 446 byte file. I don't understand as to why you need to temp datasets.

If I understand correctly , you have a 80 byte look up file and you want to take a couple of fields from this look up file and then update a 446 byte file with the values from the lookup files for all the matching keys. is that right?

Now the question is what do you want to do with the non matching keys in the 446 byte file?

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Thu Sep 23, 2004 10:08 am    Post subject: Reply with quote

Kolusu
You have got my requirement right.
Regarding Non Matching keys:
Records in the 446 byte file whose key is not present on the lookup file
should remain as they are,i.e the 156th-161st bytes and 366th byte should
contain whatever they have.(in this case it will be spaces)
Back to top
View user's profile Send private message
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Thu Sep 23, 2004 10:09 am    Post subject: Reply with quote

Oh yes Kolusu I missed 2 stmts in my toolin
My toolin looks like this

//TOOLIN DD *
SORT FROM(IN1) TO(TEMP1) USING(CTL1)
SORT FROM(IN2) TO(TEMP2) USING(CTL2)
SPLICE FROM(CON) TO(OUT1) ON(1,9,CH) WITH(156,6) -
WITH(366,1) KEEPNODUPS
SORT FROM(OUT1) TO(OUT2) USING(CTL3)
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Sat Sep 25, 2004 11:24 am    Post subject: Reply with quote

mfjin,

There's actually quite a lot wrong with your job. The main problems are (1) you need to splice using file1 as the base and file2 as the overlay, but you're doing it in reverse and (2) you're trying to use a seqnum to get the records back in their original order, but you forgot to insert the seqnum. Also, your first two operators are SORT operators, but they should be COPY operators.

Here's a DFSORT/ICETOOL job that will do what you want based on your description of what you want. You'll probably have to study it a bit to understand how it works vs your job.

Code:

//S1  EXEC PGM=ICETOOL
//TOOLMSG  DD  SYSOUT=*
//DFSMSG   DD  SYSOUT=*
//IN1 DD DSN=...  input file1
//IN2 DD DSN=...  input file2
//TEMP1 DD DSN=&&TEMP1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMP2 DD DSN=&&TEMP2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON DD DSN=*.TEMP1,VOL=REF=*.TEMP1,DISP=(OLD,PASS)
//    DD DSN=*.TEMP2,VOL=REF=*.TEMP2,DISP=(OLD,PASS)
//OUT1 DD DSN=&&O1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT2 DD DSN=...  output file
//TOOLIN DD *
* IN1->T1:  Reformat records to:
* |KEY|.......|IDSx|...|a|...|blanks|
COPY FROM(IN1) TO(TEMP1) USING(CTL1)
* IN2->T2:  Reformat records to:
* |KEY|xxx|..................|seqnum|
COPY FROM(IN2) TO(TEMP2) USING(CTL2)
* T1/T2->OUT1:  Splice to get:
* |KEY|xxx|...|IDSx|...|a|...|seqnum|
* KEY, IDSx and a come from base records (T1)
* xxx and seqnum come from overlay records (T2)
SPLICE FROM(CON) TO(OUT1) ON(1,9,CH) -
  WITHALL WITH(10,3) WITH(447,8) KEEPNODUPS
* Sort on seqnum to get the spliced records back
* in their original IN2 order.  Remove seqnum.
SORT FROM(OUT1) TO(OUT2) USING(CTL3)
/*
//CTL1CNTL DD *
  OUTREC FIELDS=(1,9,156:10,6,
          366:10,6,CHANGE=(1,C'IDSX  ',C'I',
                              C'IDSY',C'J'),
                    NOMATCH=(C' '),
                    447:8X)
/*
//CTL2CNTL DD *
  OUTREC FIELDS=(1,446,447:SEQNUM,8,ZD)
/*
//CTL3CNTL DD *
  SORT FIELDS=(447,8,ZD,A)
  OUTREC FIELDS=(1,446)
/*

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Sun Sep 26, 2004 11:24 am    Post subject: Reply with quote

Thanks Frank. I will get SPLICE right one day.
While I understood your first two observations, you say that


"Also, your first two operators are SORT operators, but they should be COPY operators"

The purpose of the first two operators are to reformat the files to enable he splicing operation. Cant SORT be used to do this?
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Sun Sep 26, 2004 2:35 pm    Post subject: Reply with quote

Well, yes you could use SORT operators instead of COPY operators to reformat the files for the SPLICE, but it would be very inefficient to do that. COPY is much more efficient than SORT. SPLICE does the SORT on 1,9,CH,A so why would you want to SORT on 1,9,CH,A twice and then do it again with SPLICE? Using two copies (COPY operators) and a sort (SPLICE operator) is much more efficient than doing three sorts (two SORT operators and a SPLICE operator). For the best performance, you should always use COPY instead of SORT when you can.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Mon Sep 27, 2004 2:38 am    Post subject: Reply with quote

Got it..Cheers Frank !!! Hopefully i shouldnt struggle for a similar job next time.
Back to top
View user's profile Send private message
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Mon Sep 27, 2004 8:34 am    Post subject: Reply with quote

Frank,
I tried this out today and I seem to be losing records after splicing.
For instance if my main file has two records with the same key that is not
present in the lookup file. ONLY ONE of them comes out in the output. Is this the result of using KEEPNODUPS ?
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Mon Sep 27, 2004 10:20 am    Post subject: Reply with quote

mfjin,

I based my solution on the data you showed me which has a match in the lookup file for every record in the main file. If the data is actually more complicated than what you showed originally, then you need to show me a more complete example of the input records and what you expect the output to look like.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Mon Sep 27, 2004 10:46 am    Post subject: Reply with quote

Ok Frank. here is the updated example

Lookup file
1-9 10-16
123456 IDSX
567015 IDSY
345678 IDSX
906615 IDSY



Main file

123456 xxa .....
30000 abc .....
30000 jhk
123456 def......
345678 fgh......
567015 nom......
345678 eti......
906615 lkg......
786345 eti.......

Output file

123456 xxa ..... IDSX I
30000 abc .....
30000 jhk......
123456 def...... IDSX I
345678 fgh...... IDSX I
567015 nom.....IDSY J
345678 eti........IDSX I
906615 lkg...... IDSY J
786345 eti.......


Now the previous job you gave me is certainly doing this.
But I get only record in the output with the non-match key 30000.The other record is getting dropped.
Please note that there are two records in the main file with key 30000.
I need both of them in my output.
Back to top
View user's profile Send private message
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Mon Sep 27, 2004 12:09 pm    Post subject: Reply with quote

Frank,
Got some idea on why this is happening

If there are duplicates in the main file having an unmatched key, SPICE tries to splice the bottom record on the top one and thereby drops the bottom record from the output. It treats it as the overlay record and the first record as the base. Ideally these records should not be spliced as they are unmatched records. Is there a way out tp prevent the the non matched duplicate records from getting spliced ?
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Mon Sep 27, 2004 12:27 pm    Post subject: Reply with quote

The reason that you're losing one of the non-matched dups is that SPLICE doesn't know it's from file2, so it splices the two dups together to get one record. To keep the non-matched dups in file2, you have to protect them by inserting an extra base record for each non-matched dup, as in the DFSORT/ICETOOL job below:

Code:

//S1  EXEC PGM=ICETOOL
//TOOLMSG  DD  SYSOUT=*
//DFSMSG   DD  SYSOUT=*
//IN1 DD DSN=...  input file1
//IN2 DD DSN=...  input file2
//TEMP1 DD DSN=&&TEMP1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMPX DD DSN=&&TEMPX,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMP2 DD DSN=&&TEMP2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON DD DSN=*.TEMP1,VOL=REF=*.TEMP1,DISP=(OLD,PASS)
//    DD DSN=*.TEMPX,VOL=REF=*.TEMPX,DISP=(OLD,PASS)
//    DD DSN=*.TEMP2,VOL=REF=*.TEMP2,DISP=(OLD,PASS)
//OUT1 DD DSN=&&O1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT2 DD DSN=...  output file
//TOOLIN DD *
* IN1->T1:  Reformat records to:
* |KEY|.......|IDSx|...|a|...|blanks|
COPY FROM(IN1) TO(TEMP1) USING(CTL1)
* IN2->TX:  Get first dup to "protect" non-matched dups
* |KEY|......................|blanks|
SELECT FROM(IN2) TO(TEMPX) ON(1,9,CH) FIRSTDUP USING(CTLX)
* IN2->T2:  Reformat records to:
* |KEY|xxx|..................|seqnum|
COPY FROM(IN2) TO(TEMP2) USING(CTL2)
* T1/T2->OUT1:  Splice to get:
* |KEY|xxx|...|IDSx|...|a|...|seqnum|
* KEY, IDSx and a come from base records (T1)
* xxx and seqnum come from overlay records (T2)
* Remove records with blank seqnum since those
* are "protect" records for matching records
* which we don't want.
SPLICE FROM(CON) TO(OUT1) ON(1,9,CH) -
  WITHALL WITH(10,3) WITH(447,8) KEEPNODUPS -
  USING(CTLY)
* Sort on seqnum to get the spliced records back
* in their original IN2 order.  Remove seqnum.
SORT FROM(OUT1) TO(OUT2) USING(CTL3)
/*
//CTL1CNTL DD *
  OUTREC FIELDS=(1,9,156:10,6,
          366:10,6,CHANGE=(1,C'IDSX  ',C'I',
                              C'IDSY',C'J'),
                    NOMATCH=(C' '),
                    447:8X)
/*
//CTLXCNTL DD *
  OUTFIL FNAMES=TEMPX,OUTREC=(1,446,447:8X)
/*
//CTL2CNTL DD *
  OUTREC FIELDS=(1,446,447:SEQNUM,8,ZD)
/*
//CTLYCNTL DD *
  OUTFIL FNAMES=OUT1,OMIT=(447,1,CH,EQ,C' ')
/*
//CTL3CNTL DD *
  SORT FIELDS=(447,8,ZD,A)
  OUTREC FIELDS=(1,446)
/*

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mfjin
Beginner


Joined: 26 Apr 2003
Posts: 94
Topics: 17

PostPosted: Fri Oct 08, 2004 2:42 am    Post subject: Reply with quote

Thanks Frank. Worked like a dream. Hopefully I shouldnt bother you the next time I use splice. Thanks again
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Fri Oct 08, 2004 9:42 am    Post subject: Reply with quote

It's no bother. I'm happy to help.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group