MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Compare files
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Thu Nov 10, 2005 6:50 am    Post subject: Compare files Reply with quote

I want to compare 2 files and write the result to a third file.
Input and output files have LRECL=5000 and FB,
I have 3 keys and have the following positions in my file
ADFLAG (pos 231,32),ADJID (957,32) STATUS (1947,32)
The keys can be maximum 32 bytes

The output should be based on the following criteria

1) If keys are the same in both files (duplicate do nothing)
2) If keys are different,then two rows to be written to file3.
Record from file 1, with the amount reversed in file 3 (I could not get
this to work with MUL,-1) and also the record from file 2
3) If record from file2 not in file1 write to file3

Code:

File1

Amount    ADFLAG   ADJID   STATUS
-------      ------        --------   --------

900.00     NNNNNN   1234567    OPEN

 
File2

Amount    ADFLAG   ADJID   STATUS
-------      ------        --------   --------

900.00     NNNNNN   1234567    BOOKED
1000.00    NNNNNN   7890123    OPEN


File 3

-900.00    NNNNNN   1234567    OPEN
900.00     NNNNNN   1234567    BOOKED
1000.00    NNNNNN   7890123    OPEN

With this job I got the desired result except point 2), mentioned above,
the part, where I have to write the record from file1 with reversed amount
into file3. This is missing in my output file.

* ADD ID OF 'V' TO FILE1 RECORDS -> T1                     
   COPY FROM(IN1) TO(T1) USING(CTL1)                       
*                                                         
* ADD ID OF 'B' TO FILE2 RECORDS -> T2                     
   COPY FROM(IN2) TO(T2) USING(CTL2)                     
*                                                         
   SPLICE FROM(CON) TO(OUT)    -                           
   ON(231,32,CH)               -                           
   ON(957,32,CH)               -                           
   ON(1947,32,CH)              -                           
     WITHEACH                  -                           
     WITH(231,32)              -                           
     WITH(957,32)              -                           
     WITH(1947,32)             -                           
     USING(CTL3) KEEPNODUPS                               
/*                                             
//CTL1CNTL DD *                               
  OUTREC FIELDS=(1,5000,5001:C'V')             
/*                                             
//CTL2CNTL DD *                               
  OUTREC FIELDS=(1,5000,5001:C'B')             
/*                                             
//CTL3CNTL DD *                               
  OUTFIL FNAMES=OUT,OMIT=(5001,1,CH,EQ,C'V')   
/*
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Thu Nov 10, 2005 8:26 am    Post subject: Reply with quote

shuko,

Run the following Job and post the output of sysout here, which will give us an idea about the type of sort product we are dealing.

Code:

//STEP0100  EXEC PGM=SORT
//SYSOUT    DD SYSOUT=*
//SORTIN    DD *
test
//SORTOUT   DD SYSOUT=*
//SYSIN     DD *                                           
  SORT FIELDS=COPY                                         
/*                                                         


Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Thu Nov 10, 2005 8:32 am    Post subject: Reply with quote

My SYSOUT
Code:

BLOCKSET     COPY  TECHNIQUE SELECTED                                         
VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE   
- CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 14:30 ON THU NOV 10, 2005
  SORT FIELDS=COPY                                                             
RECORD TYPE IS F - DATA STARTS IN POSITION 1                                   
C5-K05352 C6-Q95214 C7-K90000 C8-K05352 E9-K06751 C9-BASE   E5-K05313 E7-K90000
ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED             
SRZPIJS .STEP0100.        , INPUT LRECL = 80, BLKSIZE = 80, TYPE = FB         
MAIN STORAGE = (MAX,4194304,4181086)                                           
MAIN STORAGE ABOVE 16MB = (4103646,4103646)                                   
OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,V
OPTIONS: SIZE=4194304,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,M
OPTIONS: VIO=Y,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC
OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=
OPTIONS: TMAXLIM=4194304,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=N,DSA=0 
OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=Y,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMD
OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZ
OPTIONS: NULLOUT=RC0                                                           
BSAM ACCESS METHOD USED FOR SORTOUT                                           
BSAM ACCESS METHOD USED FOR SORTIN                                             
EF-K90000 F0-Q84357 E8-K90000                                                 
OUTPUT LRECL = 80, BLKSIZE = 80, TYPE = FB
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Thu Nov 10, 2005 11:54 am    Post subject: Reply with quote

Shuko,

I can see from your DFSORT messages that you have the Dec, 2004 DFSORT PTF, so you can use IFTHEN, OVERLAY and UFF. The DFSORT job below will do what I think you're asking for. I assumed that:
- you actually want the output sorted by your three keys (the output you show isn't sorted that way)
- your amount field is in positions 1-8
- your amount values can be right aligned for output
- you want the amount set negative for the records in file1 without a match in file2, and the amount left positive for the records in file2 without a match in file1 (your rules are a little ambiguous on this point).

Here's the DFSORT/ICETOOL job:

Code:

//S1  EXEC  PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN1 DD DSN=... input file1 (FB/5000)
//IN2 DD DSN=... input file2 (FB/5000)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(MOD,PASS)
//OUT DD DSN=...  output file (FB/5000)
//TOOLIN DD *
   COPY FROM(IN1) TO(T1) USING(CTL1)
   COPY FROM(IN2) TO(T1) USING(CTL2)
   SPLICE FROM(T1) TO(OUT)     -
   ON(231,32,CH)               -
   ON(957,32,CH)               -
   ON(1947,32,CH)              -
   WITH(1,5001)                -
   KEEPNODUPS                  -
   USING(CTL3)
/*
//CTL1CNTL DD *
  OUTREC OVERLAY=(5001:C'11')
/*
//CTL2CNTL DD *
  OUTREC OVERLAY=(5001:C'22')
/*
//CTL3CNTL DD *
  OUTFIL FNAMES=OUT,INCLUDE=(5001,2,SS,EQ,C'11,22'),
    IFTHEN=(WHEN=(5001,2,CH,EQ,C'11'),
      OVERLAY=(1:1,8,UFF,MUL,-1,EDIT=(SIIIT.TT),SIGNS=(,-))),
    IFTHEN=(WHEN=NONE,
      OVERLAY=(1:1,8,UFF,EDIT=(SIIIT.TT),SIGNS=(,-))),
      IFOUTLEN=5000
/*


OUT will have:

Code:

  Amount   ADFLAG     ADJID      STATUS
 -------   ------     --------   --------

  900.00   NNNNNN     1234567    BOOKED
 -900.00   NNNNNN     1234567    OPEN
 1000.00   NNNNNN     7890123    OPEN

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Fri Nov 11, 2005 5:29 am    Post subject: Reply with quote

Frank

Your assumptions are right. Something regarding amount, if there is no leading sign for amount, then it is assumed to be positive else it is (-) for negative. So for records in file1 without a match in file2, the Amount in file1 should be reversed, which means if there is no leading sign then the Amount is assumed to be positive so the Amount from file1 should have a leading sign (-) when it appears in out file, and if the leading sign is (-), in file1,the amount should have no leading sign. (In COBOL I would do Amount * -1, to reverse the Amount).

I tried the DFSORT/ICETOOL job and my observations were
- Duplicates appear in out file ( in my case should'nt happen)
- Records in file1 without a match in file2 and with the amount reversed
does not appear in out file


Amount ADFLAG ADJID STATUS
------- ------ -------- --------

900.00 NNNNNN 1234567 BOOKED
-900.00 NNNNNN 1234567 OPEN <--- (not in out file)
1000.00 NNNNNN 7890123 OPEN
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Fri Nov 11, 2005 11:17 am    Post subject: Reply with quote

Shuko,

You didn't show any negative values in the input file, so I assumed you didn't have any. If you do, then just change UFF to SFF in the two places it appears in my job. SFF handles positive or negative values whereas UFF only handles positive values. SFF,MUL,-1 will reverse the sign.

I don't understand your "observations". I ran the job I showed with the input you showed and it gave the output I showed:

Code:

  Amount   ADFLAG     ADJID      STATUS
 -------   ------     --------   --------

  900.00   NNNNNN     1234567    BOOKED
 -900.00   NNNNNN     1234567    OPEN
 1000.00   NNNNNN     7890123    OPEN


If you got a different output, then you either ran with different input then you showed, or you changed something in the job I showed. Check that out and if you still aren't getting the output I'm getting, show me YOUR JCL and control statements. If it looks ok, then I'll give you a job that will print out your input fields so we can see if they match my input fields.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Sat Nov 12, 2005 1:12 pm    Post subject: Reply with quote

Sorry Frank, my mistake. Embarassed I used my Test JCL and your control statements and this
resulted in the wrong output. I ran your Job and it worked like a charm for me.
Thank you. You mentioned something interesting about printing input fields.
Can you tell me how to do this. This could be really handy, during tests.
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Sat Nov 12, 2005 5:40 pm    Post subject: Reply with quote

Quote:
You mentioned something interesting about printing input fields.


Not that interesting, really. Since your fields were spread out over a large record, I was going to give you an OUTREC statement that would show just the fields of interest, e,g.

Code:

   OPTION COPY
   OUTREC FIELDS=(1,8,X,231,32,X,957,32,X,1947,32)


This would make it easier to visually see what just those fields looked like.

Anyway, I'm glad the job did what you wanted.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Wed Nov 16, 2005 10:16 am    Post subject: Reply with quote

With the above Job from Frank, the record from file1 with the amount reversed and the record from file2 unchanged, is written to file3 perfectly.
Now I have an additional requirement, when the status in file2 is SETTLED and the STATUS in file1 is OPEN or BOOKED. In this case
I want the record from file1 to be written to file3 only if the difference in Amount not equals 0.

AMOUNT ADFLAG ADJID STATUS
---------------------------------------------------------
Case1
950 N 1000000001 BOOKED (file1 )
950 N 1000000001 SETTLED (file2 )

Case2
950 N 1000000001 BOOKED (file1 )
850 N 1000000001 SETTLED (file2 )

Expected Case1 result

Since the amount from file1 gets reversed ( -950) and file 2 has +950
resulting in 0, only the record from file2 should be in file3.

AMOUNT ADFLAG ADJID STATUS
-------------------------------------------------------
950 N 1000000001 SETTLED

Expected Case2 result

-950 N 1000000001 BOOKED
850 N 1000000001 SETTLED
850 N 1000000001 BOOKED


The logic is, for every SETTLED found in File2 generate another record with the status BOOKED with the same amount as in file2,check
if the reversed amount from file1 and the generated BOOKED amount results in 0
if YES write only the record from File2 (SETTLED) to file3
if NO, then 3 records in file3 as shown in red.
I tried to get my new requriement to work, but did not have much luck.
Any help will be greatly appreciated.
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Wed Nov 16, 2005 11:13 am    Post subject: Reply with quote

Is this a completely separate requirement, or an addition to the other requirement? Please show an example of the input records and expected output that illustrates ALL of the rules you want to use.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Wed Nov 16, 2005 2:51 pm    Post subject: Reply with quote

This is an addition to the previous requirement. The files length and keys are the same (3 keys and with the following positions in my file
ADFLAG (pos 231,32),ADJID (957,32) STATUS (1947,32). The previous requirement was file1 with a status of OPEN and file 2 with a status of BOOKED, the OUT file had record from file1 with Amount reversed and the record from file2 and also any record from file2 not in file1. Now file2 can have a status of SETTLED which is shown in Case1 where the amount from file1 and file2 gives 0, and Case2 where 3 records are generated from 2 since the Amount not = 0. Since this is an additional requirement, I did not start a new topic. Have I given all the necessary information?
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Wed Nov 16, 2005 3:00 pm    Post subject: Reply with quote

Quote:
Please show an example of the input records and expected output that illustrates ALL of the rules you want to use.

_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Wed Nov 16, 2005 4:34 pm    Post subject: Reply with quote

Here are all the rules

File1

Code:
Amount     ADFLAG   ADJID       STATUS
--------------------------------------------------------------

900.00       N       1234567        OPEN    (1)
950.00       N       1000000001   BOOKED   (3)
350.00       N       3000000001    BOOKED   (4)


File2

Code:
Amount    ADFLAG   ADJID        STATUS
--------------------------------------------------------------

900.00       N      1234567      BOOKED    (1)
1000.00     N      7890123      OPEN        (2)
950.00       N     1000000001    SETTLED    (3)
250.00       N     3000000001    SETTLED    (4)


OUT file


Code:
Amount   ADFLAG     ADJID      STATUS
--------------------------------------------------------------

  900.00     N      1234567      BOOKED   (1)
 -900.00     N      1234567      OPEN     (1)
 1000.00    N       7890123      OPEN     (2)
 950.00      N      1000000001   SETTLED  (3)
 -350.00     N      3000000001   BOOKED   (4)
 250.00      N      3000000001   BOOKED   (4)
 250.00      N      3000000001   SETTLED  (4)


Rule 1: and Rule 2 works with the Job you posted.
Rule 3: the same job works but I don't want the record
from file1 in my out file as the reversed amount -950 from file1 and the
Amount from file2 gives 0.
Rule 4: Since there is a difference of 100 in the Amount,3 records in the out file
Rule3 and 4 should be applicable whenver the Status is SETTLED in File2
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose

PostPosted: Wed Nov 16, 2005 9:39 pm    Post subject: Reply with quote

I just had an idea for a much less complex way to do this kind of thing. I'll see if I can work it out sometime tomorrow. Stay tuned.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort


Last edited by Frank Yaeger on Fri Nov 18, 2005 10:45 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20

PostPosted: Thu Nov 17, 2005 2:37 am    Post subject: Reply with quote

Sorry Frank for underestimating the complexity of my requirement.
I was trying to integrate my additional requirement, with the posted job
without losing this functionality. Additionaly I just want to check the
status of file2 and if it is SETTLED, check the AMOUNT in File1 (taking into
consideration it is reversed) and FIle2, if this gives 0,discard the row from
file1 and if it is not = 0, then record from file1 with reversed Amount in
file3,record from file2 in file3, the posted job does this and now addtionally
I need the record from file2 again in file 3 the same AMOUNT but only
the Status as BOOKED.
I was trying to figure out a way to eliminate the file1 record from OUT file,if amount = 0 and add an addtional record to out file if amount not=0 by checking the STATUS in file2 for SETTLED ,to the existing functional job. I thought this would only need some tweaking to the existing Job.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group