Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Thu Sep 23, 2004 10:01 am Post subject:
Mfjin,
I cannot relate your JOb to your output.You are missing a copy statement in the TOOLIN for formating the lookup(80 byte) file to 446 byte file. I don't understand as to why you need to temp datasets.
If I understand correctly , you have a 80 byte look up file and you want to take a couple of fields from this look up file and then update a 446 byte file with the values from the lookup files for all the matching keys. is that right?
Now the question is what do you want to do with the non matching keys in the 446 byte file?
Kolusu
You have got my requirement right.
Regarding Non Matching keys:
Records in the 446 byte file whose key is not present on the lookup file
should remain as they are,i.e the 156th-161st bytes and 366th byte should
contain whatever they have.(in this case it will be spaces)
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Sat Sep 25, 2004 11:24 am Post subject:
mfjin,
There's actually quite a lot wrong with your job. The main problems are (1) you need to splice using file1 as the base and file2 as the overlay, but you're doing it in reverse and (2) you're trying to use a seqnum to get the records back in their original order, but you forgot to insert the seqnum. Also, your first two operators are SORT operators, but they should be COPY operators.
Here's a DFSORT/ICETOOL job that will do what you want based on your description of what you want. You'll probably have to study it a bit to understand how it works vs your job.
Code:
//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN1 DD DSN=... input file1
//IN2 DD DSN=... input file2
//TEMP1 DD DSN=&&TEMP1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMP2 DD DSN=&&TEMP2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON DD DSN=*.TEMP1,VOL=REF=*.TEMP1,DISP=(OLD,PASS)
// DD DSN=*.TEMP2,VOL=REF=*.TEMP2,DISP=(OLD,PASS)
//OUT1 DD DSN=&&O1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT2 DD DSN=... output file
//TOOLIN DD *
* IN1->T1: Reformat records to:
* |KEY|.......|IDSx|...|a|...|blanks|
COPY FROM(IN1) TO(TEMP1) USING(CTL1)
* IN2->T2: Reformat records to:
* |KEY|xxx|..................|seqnum|
COPY FROM(IN2) TO(TEMP2) USING(CTL2)
* T1/T2->OUT1: Splice to get:
* |KEY|xxx|...|IDSx|...|a|...|seqnum|
* KEY, IDSx and a come from base records (T1)
* xxx and seqnum come from overlay records (T2)
SPLICE FROM(CON) TO(OUT1) ON(1,9,CH) -
WITHALL WITH(10,3) WITH(447,8) KEEPNODUPS
* Sort on seqnum to get the spliced records back
* in their original IN2 order. Remove seqnum.
SORT FROM(OUT1) TO(OUT2) USING(CTL3)
/*
//CTL1CNTL DD *
OUTREC FIELDS=(1,9,156:10,6,
366:10,6,CHANGE=(1,C'IDSX ',C'I',
C'IDSY',C'J'),
NOMATCH=(C' '),
447:8X)
/*
//CTL2CNTL DD *
OUTREC FIELDS=(1,446,447:SEQNUM,8,ZD)
/*
//CTL3CNTL DD *
SORT FIELDS=(447,8,ZD,A)
OUTREC FIELDS=(1,446)
/*
_________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Sun Sep 26, 2004 2:35 pm Post subject:
Well, yes you could use SORT operators instead of COPY operators to reformat the files for the SPLICE, but it would be very inefficient to do that. COPY is much more efficient than SORT. SPLICE does the SORT on 1,9,CH,A so why would you want to SORT on 1,9,CH,A twice and then do it again with SPLICE? Using two copies (COPY operators) and a sort (SPLICE operator) is much more efficient than doing three sorts (two SORT operators and a SPLICE operator). For the best performance, you should always use COPY instead of SORT when you can. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Frank,
I tried this out today and I seem to be losing records after splicing.
For instance if my main file has two records with the same key that is not
present in the lookup file. ONLY ONE of them comes out in the output. Is this the result of using KEEPNODUPS ?
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Mon Sep 27, 2004 10:20 am Post subject:
mfjin,
I based my solution on the data you showed me which has a match in the lookup file for every record in the main file. If the data is actually more complicated than what you showed originally, then you need to show me a more complete example of the input records and what you expect the output to look like. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
123456 xxa ..... IDSX I
30000 abc .....
30000 jhk......
123456 def...... IDSX I
345678 fgh...... IDSX I
567015 nom.....IDSY J
345678 eti........IDSX I
906615 lkg...... IDSY J
786345 eti.......
Now the previous job you gave me is certainly doing this.
But I get only record in the output with the non-match key 30000.The other record is getting dropped.
Please note that there are two records in the main file with key 30000.
I need both of them in my output.
If there are duplicates in the main file having an unmatched key, SPICE tries to splice the bottom record on the top one and thereby drops the bottom record from the output. It treats it as the overlay record and the first record as the base. Ideally these records should not be spliced as they are unmatched records. Is there a way out tp prevent the the non matched duplicate records from getting spliced ?
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Mon Sep 27, 2004 12:27 pm Post subject:
The reason that you're losing one of the non-matched dups is that SPLICE doesn't know it's from file2, so it splices the two dups together to get one record. To keep the non-matched dups in file2, you have to protect them by inserting an extra base record for each non-matched dup, as in the DFSORT/ICETOOL job below:
Code:
//S1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN1 DD DSN=... input file1
//IN2 DD DSN=... input file2
//TEMP1 DD DSN=&&TEMP1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMPX DD DSN=&&TEMPX,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//TEMP2 DD DSN=&&TEMP2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//CON DD DSN=*.TEMP1,VOL=REF=*.TEMP1,DISP=(OLD,PASS)
// DD DSN=*.TEMPX,VOL=REF=*.TEMPX,DISP=(OLD,PASS)
// DD DSN=*.TEMP2,VOL=REF=*.TEMP2,DISP=(OLD,PASS)
//OUT1 DD DSN=&&O1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT2 DD DSN=... output file
//TOOLIN DD *
* IN1->T1: Reformat records to:
* |KEY|.......|IDSx|...|a|...|blanks|
COPY FROM(IN1) TO(TEMP1) USING(CTL1)
* IN2->TX: Get first dup to "protect" non-matched dups
* |KEY|......................|blanks|
SELECT FROM(IN2) TO(TEMPX) ON(1,9,CH) FIRSTDUP USING(CTLX)
* IN2->T2: Reformat records to:
* |KEY|xxx|..................|seqnum|
COPY FROM(IN2) TO(TEMP2) USING(CTL2)
* T1/T2->OUT1: Splice to get:
* |KEY|xxx|...|IDSx|...|a|...|seqnum|
* KEY, IDSx and a come from base records (T1)
* xxx and seqnum come from overlay records (T2)
* Remove records with blank seqnum since those
* are "protect" records for matching records
* which we don't want.
SPLICE FROM(CON) TO(OUT1) ON(1,9,CH) -
WITHALL WITH(10,3) WITH(447,8) KEEPNODUPS -
USING(CTLY)
* Sort on seqnum to get the spliced records back
* in their original IN2 order. Remove seqnum.
SORT FROM(OUT1) TO(OUT2) USING(CTL3)
/*
//CTL1CNTL DD *
OUTREC FIELDS=(1,9,156:10,6,
366:10,6,CHANGE=(1,C'IDSX ',C'I',
C'IDSY',C'J'),
NOMATCH=(C' '),
447:8X)
/*
//CTLXCNTL DD *
OUTFIL FNAMES=TEMPX,OUTREC=(1,446,447:8X)
/*
//CTL2CNTL DD *
OUTREC FIELDS=(1,446,447:SEQNUM,8,ZD)
/*
//CTLYCNTL DD *
OUTFIL FNAMES=OUT1,OMIT=(447,1,CH,EQ,C' ')
/*
//CTL3CNTL DD *
SORT FIELDS=(447,8,ZD,A)
OUTREC FIELDS=(1,446)
/*
_________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Fri Oct 08, 2004 9:42 am Post subject:
It's no bother. I'm happy to help. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum