I would like to know how above 2 can be achived. I can get only the input file. Can we achive the file split even if we don't know how many recs the input file has ?
Thanx for the kind reply .The first method works fine but i want to know that how can we split the file in the second way ( not in round robin fashion) without knowing how many records are there in that file .
Joined: 26 Nov 2002 Posts: 12372 Topics: 75 Location: San Jose
Posted: Wed Nov 05, 2003 3:43 am Post subject:
seema,
The above suggested method works even if we don't the know the no: of records in the file.The first 3 files will have 250 records each(assuming that your inout file has more than 750). The last file will have rest of the records.
please clarify us the following questions.
1. How many files do you need the input file split up?
2. How many records do you want in each file?
3. what is the LRECL & RECFM
If you want dynamic split it can be done but I need clear requirements.
I wanted to split the file equally, but i will not be knowing how many records the input file can contain.
To begin with, I wanted to know how a file could be split into 4 files. If number split could be decided dynamically, it will add value to my design. The Lrecl is 80 and record format is FB.
Joined: 26 Nov 2002 Posts: 12372 Topics: 75 Location: San Jose
Posted: Wed Nov 05, 2003 6:48 am Post subject:
seema,
The equal splitting of the files will not always be true. If the total no of records is a multiple of the number of files to be split then you will have equal no: of records in all the files. But if it is not then the last file will have more records.
The following DFSORT/ICETOOL JCl will give you the desired results. This Job uses the horizontal math fucntions which are available in the new PTF UQ90053. If you have syncsort at your shop then change the pgm name to Synctool.
I assumed that you want to split the input file into 4 output files. This solution also assumes that you have a min of 4 records in the input file.
A brief explanation of the Job.We first take the input file and add a seqnum to it and sort on it descending. By doing so you will get last record at the beginning of the file. Now we take this value and do our manipulation to create dynamic control cards with startrec and endrec values.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Nov 05, 2003 10:36 am Post subject:
Kolusu,
You forgot the TOOLIN statements. Also, you can use SAVE for the last OUTFIL statement instead of generating STARTREC for it.
For a large file, it might be more efficient to get the count in one COPY operator and generate the control statements from that count in a second COPY operator, rather than doing both in a SORT operator. The COPY would be more efficient than the SORT. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12372 Topics: 75 Location: San Jose
Posted: Wed Nov 05, 2003 11:06 am Post subject:
Frank,
Thanks for catching error. I am editing the post to show the toolin statements. A copy operator would be more efficient if we are dealing with a large file, but will need an extra pass to format the count from the trailer.
I was under the impression that SAVE can be used along with INCLUDE/OMIT conditions. Using it along with startrec/endrec might not give the desired results.
Kolusu
PS: I will post the solution using copy operator later.
If you are using syncsort and you get an error on REMOVECC parameter then you are having an old version of syncsort. In that case use the following control cards.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Wed Nov 05, 2003 11:49 am Post subject:
Kolusu,
{Added after I saw your post with the second job}:
I wrote this post before you posted your second job which looks a lot like mine. I'm referring to your first job here:
DFSORT can use SAVE with STARTREC and ENDREC. Any records not included by the STARTREC and ENDREC parameters of the other OUTFIL statements will be included for the OUTFIL statement with SAVE.
You only have a SORT operator in your TOOLIN. That will generate the statements in CTL2CNTL, but you need a COPY operator to use the CTL2CNTL statements. You should have:
I copy all of the records to generate the count record, copy that one record to generate the control statements, and copy all of the records to split the file. Thus, you're doing a sort of all the records and a copy of all of the records, whereas I'm doing a copy of all the records, a copy of one record, and a copy of all the records. The two copies of all the records is faster than the sort and copy of all the records.
I ran our two jobs using 500001 records. Mine ran in less than half the CPU time of yours. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12372 Topics: 75 Location: San Jose
Posted: Wed Nov 05, 2003 11:56 am Post subject:
Frank,
Unfortunately Syncosrt does not allow edit masking on trailer record. It is 8 byte field with leading zeroes supressed. So I had to an extra inrec statement to mask the count from FS to ZD field.
Thanks for running the 2 jobs and posting the results.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Quick Manuals
Register
Profile
Log in to check your private messages
Log in
