Posted: Thu Nov 10, 2005 6:50 am Post subject: Compare files
I want to compare 2 files and write the result to a third file.
Input and output files have LRECL=5000 and FB,
I have 3 keys and have the following positions in my file
ADFLAG (pos 231,32),ADJID (957,32) STATUS (1947,32)
The keys can be maximum 32 bytes
The output should be based on the following criteria
1) If keys are the same in both files (duplicate do nothing)
2) If keys are different,then two rows to be written to file3.
Record from file 1, with the amount reversed in file 3 (I could not get
this to work with MUL,-1) and also the record from file 2
3) If record from file2 not in file1 write to file3
Code:
File1
Amount ADFLAG ADJID STATUS
------- ------ -------- --------
900.00 NNNNNN 1234567 OPEN
File2
Amount ADFLAG ADJID STATUS
------- ------ -------- --------
900.00 NNNNNN 1234567 BOOKED
1000.00 NNNNNN 7890123 OPEN
File 3
-900.00 NNNNNN 1234567 OPEN
900.00 NNNNNN 1234567 BOOKED
1000.00 NNNNNN 7890123 OPEN
With this job I got the desired result except point 2), mentioned above,
the part, where I have to write the record from file1 with reversed amount
into file3. This is missing in my output file.
* ADD ID OF 'V' TO FILE1 RECORDS -> T1
COPY FROM(IN1) TO(T1) USING(CTL1)
*
* ADD ID OF 'B' TO FILE2 RECORDS -> T2
COPY FROM(IN2) TO(T2) USING(CTL2)
*
SPLICE FROM(CON) TO(OUT) -
ON(231,32,CH) -
ON(957,32,CH) -
ON(1947,32,CH) -
WITHEACH -
WITH(231,32) -
WITH(957,32) -
WITH(1947,32) -
USING(CTL3) KEEPNODUPS
/*
//CTL1CNTL DD *
OUTREC FIELDS=(1,5000,5001:C'V')
/*
//CTL2CNTL DD *
OUTREC FIELDS=(1,5000,5001:C'B')
/*
//CTL3CNTL DD *
OUTFIL FNAMES=OUT,OMIT=(5001,1,CH,EQ,C'V')
/*
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Thu Nov 10, 2005 11:54 am Post subject:
Shuko,
I can see from your DFSORT messages that you have the Dec, 2004 DFSORT PTF, so you can use IFTHEN, OVERLAY and UFF. The DFSORT job below will do what I think you're asking for. I assumed that:
- you actually want the output sorted by your three keys (the output you show isn't sorted that way)
- your amount field is in positions 1-8
- your amount values can be right aligned for output
- you want the amount set negative for the records in file1 without a match in file2, and the amount left positive for the records in file2 without a match in file1 (your rules are a little ambiguous on this point).
Amount ADFLAG ADJID STATUS
------- ------ -------- --------
900.00 NNNNNN 1234567 BOOKED
-900.00 NNNNNN 1234567 OPEN
1000.00 NNNNNN 7890123 OPEN
_________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Your assumptions are right. Something regarding amount, if there is no leading sign for amount, then it is assumed to be positive else it is (-) for negative. So for records in file1 without a match in file2, the Amount in file1 should be reversed, which means if there is no leading sign then the Amount is assumed to be positive so the Amount from file1 should have a leading sign (-) when it appears in out file, and if the leading sign is (-), in file1,the amount should have no leading sign. (In COBOL I would do Amount * -1, to reverse the Amount).
I tried the DFSORT/ICETOOL job and my observations were
- Duplicates appear in out file ( in my case should'nt happen)
- Records in file1 without a match in file2 and with the amount reversed
does not appear in out file
Amount ADFLAG ADJID STATUS
------- ------ -------- --------
900.00 NNNNNN 1234567 BOOKED
-900.00 NNNNNN 1234567 OPEN <--- (not in out file)
1000.00 NNNNNN 7890123 OPEN
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Fri Nov 11, 2005 11:17 am Post subject:
Shuko,
You didn't show any negative values in the input file, so I assumed you didn't have any. If you do, then just change UFF to SFF in the two places it appears in my job. SFF handles positive or negative values whereas UFF only handles positive values. SFF,MUL,-1 will reverse the sign.
I don't understand your "observations". I ran the job I showed with the input you showed and it gave the output I showed:
Code:
Amount ADFLAG ADJID STATUS
------- ------ -------- --------
900.00 NNNNNN 1234567 BOOKED
-900.00 NNNNNN 1234567 OPEN
1000.00 NNNNNN 7890123 OPEN
If you got a different output, then you either ran with different input then you showed, or you changed something in the job I showed. Check that out and if you still aren't getting the output I'm getting, show me YOUR JCL and control statements. If it looks ok, then I'll give you a job that will print out your input fields so we can see if they match my input fields. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Sorry Frank, my mistake. I used my Test JCL and your control statements and this
resulted in the wrong output. I ran your Job and it worked like a charm for me.
Thank you. You mentioned something interesting about printing input fields.
Can you tell me how to do this. This could be really handy, during tests.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Sat Nov 12, 2005 5:40 pm Post subject:
Quote:
You mentioned something interesting about printing input fields.
Not that interesting, really. Since your fields were spread out over a large record, I was going to give you an OUTREC statement that would show just the fields of interest, e,g.
This would make it easier to visually see what just those fields looked like.
Anyway, I'm glad the job did what you wanted. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
With the above Job from Frank, the record from file1 with the amount reversed and the record from file2 unchanged, is written to file3 perfectly.
Now I have an additional requirement, when the status in file2 is SETTLED and the STATUS in file1 is OPEN or BOOKED. In this case
I want the record from file1 to be written to file3 only if the difference in Amount not equals 0.
AMOUNT ADFLAG ADJID STATUS
---------------------------------------------------------
Case1
950 N 1000000001 BOOKED (file1 )
950 N 1000000001 SETTLED (file2 )
Case2
950 N 1000000001 BOOKED (file1 )
850 N 1000000001 SETTLED (file2 )
Expected Case1 result
Since the amount from file1 gets reversed ( -950) and file 2 has +950
resulting in 0, only the record from file2 should be in file3.
AMOUNT ADFLAG ADJID STATUS
-------------------------------------------------------
950 N 1000000001 SETTLED
Expected Case2 result
-950 N 1000000001 BOOKED
850 N 1000000001 SETTLED
850 N 1000000001 BOOKED
The logic is, for every SETTLED found in File2 generate another record with the status BOOKED with the same amount as in file2,check
if the reversed amount from file1 and the generated BOOKED amount results in 0
if YES write only the record from File2 (SETTLED) to file3
if NO, then 3 records in file3 as shown in red.
I tried to get my new requriement to work, but did not have much luck.
Any help will be greatly appreciated.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Nov 16, 2005 11:13 am Post subject:
Is this a completely separate requirement, or an addition to the other requirement? Please show an example of the input records and expected output that illustrates ALL of the rules you want to use. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
This is an addition to the previous requirement. The files length and keys are the same (3 keys and with the following positions in my file
ADFLAG (pos 231,32),ADJID (957,32) STATUS (1947,32). The previous requirement was file1 with a status of OPEN and file 2 with a status of BOOKED, the OUT file had record from file1 with Amount reversed and the record from file2 and also any record from file2 not in file1. Now file2 can have a status of SETTLED which is shown in Case1 where the amount from file1 and file2 gives 0, and Case2 where 3 records are generated from 2 since the Amount not = 0. Since this is an additional requirement, I did not start a new topic. Have I given all the necessary information?
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Nov 16, 2005 3:00 pm Post subject:
Quote:
Please show an example of the input records and expected output that illustrates ALL of the rules you want to use.
_________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Amount ADFLAG ADJID STATUS
--------------------------------------------------------------
900.00 N 1234567 OPEN (1)
950.00 N 1000000001 BOOKED (3)
350.00 N 3000000001 BOOKED (4)
File2
Code:
Amount ADFLAG ADJID STATUS
--------------------------------------------------------------
900.00 N 1234567 BOOKED (1)
1000.00 N 7890123 OPEN (2)
950.00 N 1000000001 SETTLED (3)
250.00 N 3000000001 SETTLED (4)
OUT file
Code:
Amount ADFLAG ADJID STATUS
--------------------------------------------------------------
900.00 N 1234567 BOOKED (1)
-900.00 N 1234567 OPEN (1)
1000.00 N 7890123 OPEN (2)
950.00 N 1000000001 SETTLED (3)
-350.00 N 3000000001 BOOKED (4)
250.00 N 3000000001 BOOKED (4)
250.00 N 3000000001 SETTLED (4)
Rule 1: and Rule 2 works with the Job you posted.
Rule 3: the same job works but I don't want the record
from file1 in my out file as the reversed amount -950 from file1 and the
Amount from file2 gives 0.
Rule 4: Since there is a difference of 100 in the Amount,3 records in the out file
Rule3 and 4 should be applicable whenver the Status is SETTLED in File2
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Nov 16, 2005 9:39 pm Post subject:
I just had an idea for a much less complex way to do this kind of thing. I'll see if I can work it out sometime tomorrow. Stay tuned. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Last edited by Frank Yaeger on Fri Nov 18, 2005 10:45 am; edited 1 time in total
Sorry Frank for underestimating the complexity of my requirement.
I was trying to integrate my additional requirement, with the posted job
without losing this functionality. Additionaly I just want to check the
status of file2 and if it is SETTLED, check the AMOUNT in File1 (taking into
consideration it is reversed) and FIle2, if this gives 0,discard the row from
file1 and if it is not = 0, then record from file1 with reversed Amount in
file3,record from file2 in file3, the posted job does this and now addtionally
I need the record from file2 again in file 3 the same AMOUNT but only
the Status as BOOKED.
I was trying to figure out a way to eliminate the file1 record from OUT file,if amount = 0 and add an addtional record to out file if amount not=0 by checking the STATUS in file2 for SETTLED ,to the existing functional job. I thought this would only need some tweaking to the existing Job.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum