MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Split header and trailer file into several files.
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities
View previous topic :: View next topic  
Author Message
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Mon Aug 20, 2007 7:02 am    Post subject: Split header and trailer file into several files. Reply with quote
I have a file with header and trailer records.
The header records always starts with 1 in postion 1 and trailer records with 2 in position 2. The file is FB with LRECL=68.

Example of a file with header and trailer records.

Code:

1Header1                         xxxxxxxxxxxxxxxxxxx             x
2aaaaa                           bbbbbb                          X
2bbbbb                           20030401                        S
2........                        ....                            .
1Header2                         xxxxxxxxxxxxxxxxxxx             x
2aaaaa                           bbbbbb                          X
2bbbbb                           20030401                        S
2gggggggg                        0                               S
2........                        ....                            .

and so on.

The file will have a about a million records. I want to split this file into 10 files.
The first 10,000 header records(with 1 in postion 1) and the corresponding trailer records(with 2 in position 2)
should be in the first file and the next 10,000 header records(with 1 in postion 1) and the corresponding trailer records(with 2 in position 2) should be in the second file and so on and the last bit which could be less than 10,000 in the last file.

I have seen the split files examples on this board. In my case I have to make sure that the header records has all it's trailer records in the same file.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose
PostPosted: Mon Aug 20, 2007 7:18 am    Post subject: Reply with quote
Quote:

The header records always starts with 1 in postion 1 and trailer records with 2 in position 2.


Shuko,

The sample data you have shown does not match the description. I see that the detail records also have a '2' in the first position. Also when you say you want extract header and trailer , do you mean you just want the header and trailer without the detail records like shown below?

ex:
Code:

1Header1   
 2trailer1
1Header2   
 2trailer2
1Header3   
 2trailer3
..
1Header10,000   
 2trailer10,000


Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Mon Aug 20, 2007 2:15 pm    Post subject: Reply with quote
Kolusu

Sorry my mistake. By trailer I actually wanted to say detail records.
so the first output file should have the following output,10,000 times header record with the detail records and the next file with the next 10,000 and so on.

The first output file

Code:

1Header1                         xxxxxxxxxxxxxxxxxxx             x
2aaaaa                           bbbbbb                          X
2bbbbb                           20030401                        S
2........                        ....                            .
1Header2                         xxxxxxxxxxxxxxxxxxx             x
2aaaaa                           bbbbbb                          X
2bbbbb                           20030401                        S
2gggggggg                        0                               S
2........                        ....                            .

and so on till 10,000 times header with the detail records


Shuko
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose
PostPosted: Mon Aug 20, 2007 2:46 pm    Post subject: Reply with quote
shuko,

try this JCL. I just showed first 4 output file control cards . You can add for other files

Code:

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=* 
//SORTIN   DD DSN=your input file,
//            DISP=SHR
//OUT01    DD DSN=your output file01,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
/*
//OUT02    DD DSN=your output file02,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
/*
//OUT03    DD DSN=your output file03,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
/*
//OUT04    DD DSN=your output file04,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
/*
...

//OUT10    DD DSN=your output file10,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
/*
//SYSIN    DD *         
  SORT FIELDS=COPY                                     
  OUTREC IFTHEN=(WHEN=INIT,                           
        OVERLAY=(69:SEQNUM,8,ZD)),                     
                                                       
         IFTHEN=(WHEN=(1,1,CH,NE,C'1'),               
        OVERLAY=(77:SEQNUM,8,ZD,                       
                 69:69,8,ZD,SUB,77,8,ZD,M11,LENGTH=8)),
                                                       
         IFTHEN=(WHEN=NONE,                           
        OVERLAY=(69:SEQNUM,8,ZD))                     
                                                       
  OUTFIL FNAMES=OUT01,OUTREC=(1,68),
  INCLUDE=(69,8,ZD,LE,10000)
                           
  OUTFIL FNAMES=OUT02,OUTREC=(1,68),
  INCLUDE=(69,8,ZD,GE,10001,AND,69,8,ZD,LE,20000)

  OUTFIL FNAMES=OUT03,OUTREC=(1,68),
  INCLUDE=(69,8,ZD,GE,20001,AND,69,8,ZD,LE,30000)

  OUTFIL FNAMES=OUT04,OUTREC=(1,68),
  INCLUDE=(69,8,ZD,GE,30001,AND,69,8,ZD,LE,40000)

  .....

/*


Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Tue Aug 21, 2007 12:51 am    Post subject: Reply with quote
Thank you Kolusu. The JCL worked as expected.
I was just wondering, if it is possible to averagely divide the input file into 10 files, since I won't know the exact size of the file in advance.
Suppose I have 175,000 times, Header records with trailer records in the input file. Split this into 10 files and write 17,500 in each file. Is this possible?

Shuko
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose
PostPosted: Tue Aug 21, 2007 7:19 am    Post subject: Reply with quote
shuko,

bonk bonk is it really hard that hard to describe your problem in one go? The dynamic split has been discussed many times in this forum. Please SEARCH before posting. Check this topic.

http://www.mvsforums.com/helpboards/viewtopic.php?t=5618&highlight=split+dynamic

In the same topic the solutions are listed , however I will make it easy for you

http://www.mvsforums.com/helpboards/viewtopic.php?p=31020#31020

http://www.mvsforums.com/helpboards/viewtopic.php?p=27519#27519

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Tue Aug 21, 2007 9:26 am    Post subject: Reply with quote
Thank you Kolusu

Shuko
Back to top
View user's profile Send private message
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Fri Oct 26, 2007 9:06 am    Post subject: Reply with quote
I seem to have a problem now with splitting header and detail records into several files. I am trying to split a file with several million records.
with this code. I am splitting the source file into 5 files of 500,000 each.
This 500,000 is the header with the detail records. I had a problem only in the fifth file. The first few records in the fifth file had the header and the detail records and then the following records had only the header record without the detail records. This happened
only in the last file. What am I doing wrong here?. For smaller files, this code works well.

Code:

SORT FIELDS=COPY
OUTREC IFTHEN=(WHEN=INIT,
      OVERLAY=(69:SEQNUM,8,ZD)),
       IFTHEN=(WHEN=(1,1,CH,NE,C'1'),
      OVERLAY=(77:SEQNUM,8,ZD,
               69:69,8,ZD,SUB,77,8,ZD,M11,LENGTH=8)),
       IFTHEN=(WHEN=NONE,
      OVERLAY=(69:SEQNUM,8,ZD))
OUTFIL FNAMES=OUT01,OUTREC=(1,68),
INCLUDE=(69,8,ZD,LE,500000)
OUTFIL FNAMES=OUT02,OUTREC=(1,68),
INCLUDE=(69,8,ZD,GE,500001,AND,69,8,ZD,LE,1000000)
OUTFIL FNAMES=OUT03,OUTREC=(1,68),
INCLUDE=(69,8,ZD,GE,1000001,AND,69,8,ZD,LE,1500000)
OUTFIL FNAMES=OUT04,OUTREC=(1,68),
INCLUDE=(69,8,ZD,GE,1500001,AND,69,8,ZD,LE,2000000)
OUTFIL FNAMES=OUT05,OUTREC=(1,68),
INCLUDE=(69,8,ZD,GE,2000001,AND,69,8,ZD,LE,2500000)
OUTFIL FNAMES=OUT06,OUTREC=(1,68),
INCLUDE=(69,8,ZD,GE,2500001,AND,69,8,ZD,LE,3000000)



shuko
Back to top
View user's profile Send private message
vivek1983
Intermediate


Joined: 20 Apr 2006
Posts: 222
Topics: 24
PostPosted: Fri Oct 26, 2007 9:36 am    Post subject: Reply with quote
shuko,

Quote:

I am splitting the source file into 5 files of 500,000 each.


Y are there output files till OUT06 when you are splitting the input file to 5 output files?
_________________
Vivek G
--------------------------------------
A dream is just a dream. A goal is a dream with a plan and a deadline. (Harvey Mackay)
Back to top
View user's profile Send private message
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Fri Oct 26, 2007 10:43 am    Post subject: Reply with quote
Sorry, my mistake. I wanted to write 6 files of 500,00 each
and not 5.

Shuko
Back to top
View user's profile Send private message
vivek1983
Intermediate


Joined: 20 Apr 2006
Posts: 222
Topics: 24
PostPosted: Fri Oct 26, 2007 11:06 am    Post subject: Reply with quote
shuko,

Quote:

The first few records in the fifth file had the header and the detail records and then the following records had only the header record without the detail records. This happened



Can you post the total number or records in the input file?
_________________
Vivek G
--------------------------------------
A dream is just a dream. A goal is a dream with a plan and a deadline. (Harvey Mackay)
Back to top
View user's profile Send private message
vivek1983
Intermediate


Joined: 20 Apr 2006
Posts: 222
Topics: 24
PostPosted: Fri Oct 26, 2007 11:09 am    Post subject: Reply with quote
shuko,


You can get the count of the records using TALLY option in Fileaid.

In the file aid menu, go to "Interactive option" and then issue the command TALLY command.
_________________
Vivek G
--------------------------------------
A dream is just a dream. A goal is a dream with a plan and a deadline. (Harvey Mackay)
Back to top
View user's profile Send private message
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Fri Oct 26, 2007 11:40 am    Post subject: Reply with quote
The input file had 100769064 records.

shuko
Back to top
View user's profile Send private message
Frank Yaeger
Sort Forum Moderator
Sort Forum Moderator


Joined: 02 Dec 2002
Posts: 1618
Topics: 31
Location: San Jose
PostPosted: Fri Oct 26, 2007 11:41 am    Post subject: Reply with quote
Quote:
I had a problem only in the fifth file. The first few records in the fifth file had the header and the detail records and then the following records had only the header record without the detail records.


I can't see any obvious reason why that would happen unless those last groups of records only had a header record without detail records. The job assigns a sequence number to each group of header and detail records and then you select those groups by their sequence number. So there's nothing different about the way the OUTFIL for OUT06 works vs the other OUTFILs. The difference might be in the input file.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shuko
Beginner


Joined: 08 Nov 2005
Posts: 73
Topics: 20
PostPosted: Sat Oct 27, 2007 6:38 am    Post subject: Reply with quote
Frank
This was my first reaction, regarding the input file. The input file had the header and detail records. Since the last file was corrupted, I had to manually locate the record position in the source file and use SKIPREC to copy the records for the last file. The file copied had both the header and
detail records. My tests show that, copying 500,000 header and detail records (average of 45 detail records for every header record) in four files works as expected but the fifth file does not. After a few 100 header-detail records, only the header gets copied.
I tried with a smaller file of 73840416 records. This produced 2 files with
500,000 header-detail records and the 3rd file with 488197 header-detail
records. I had both the header and detail records in the last file.

shuko
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Utilities All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group