MVSFORUMS.com

[videlord]

There are n input files, each file with m records.
Is there a way to combine nth field of the nth input file to one output file?
For example:

input1:
1 aaa xxx xxx
2 bbb xxx xxx
3 ccc xxx xxx

input2:
1 xxx 111 xxx
2 xxx 222 xxx
3 xxx 333 xxx

input3:
1 xxx xxx AAA
2 xxx xxx BBB
3 xxx xxx CCC

the output will be:
1 aaa 111 AAA
2 bbb 222 BBB
3 ccc 333 CCC

Using ICETOL "splice witheach", we can get the result.
It will sort n * m records by seqnum.
If m is too large, it will take a long long time.
Is there an more efficient way?

Thanks.

[kolusu]

videlord,

You can use Easytrieve to acehieve the desired results and it can done in one pass of the data.

Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu

[Frank Yaeger]

[videlord]

Thanks kolusu.
I will search about Easytrieve. But if it's not a product of IBM, I will not use it.

Frank,
I'm still testing the logic.
We would have 600 more millions record, and total 17 files need to be processed. I think it will take a long time.
Fisrt, expand each file to same format (Add space)
Then SPLICE 600m * 17 records.

I'm writing a PL/I program to compare with DF/SORT.

I will post the testing result later.

[Mervyn]

My money's on DFSORT
_________________
The day you stop learning the dinosaur becomes extinct

[Frank Yaeger]

videlord,

I assumed from your example that you already had the files set up for splicing. If you need to do the additional COPY runs to get them set up for splicing, that will certainly affect the total time required.

If you're going to do a performance comparison, I'd like to see the DFSORT/ICETOOL job you use to make sure it's coded "correctly".
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

[kolusu]

Meryvn,

If the files are not setup for splicing , then I think a program will be a best option as the merging is done in one pass.

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu

[videlord]

Frank,

Actually, the input files are generated from previous DFSORT jobs.
If I use SPLICE next step, I will expand the fields padded with space.
If I use PL/I program, then only sequenc number and one field needed in each files.

ICETOOL stament:
TOOLIN:
SPLICE FROM(CONCT) TO(OUT) ON(1,9,CH) WITHEACH -
WITH(xx,xx) ... WITH(xx,xx) USING(CTL1)
CTL1CNTL:
OPTION EQUALS
OUTFIL FNAMES=OUT,OUREC=(1,xxx)

[Frank Yaeger]

So you are only comparing the SPLICE operator to the PL/I program, and not the multiple COPY operators + the SPLICE operator to the PL/I program ... right?

Are you going to have your PL/I program avoid sorting by reading one record from each file in turn? If so, then it will have an advantage over SPLICE which has to sort the concatenated input files. (I'd like to allow SPLICE to work without sorting, when appropriate, in the future, but it can't do that now.)

You don't need OPTION EQUALS ... SPLICE uses it automatically.
"OUREC" should be "OUTREC".
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

[Frank Yaeger]

Hmmm ... why do you need the OUTFIL statement? If you pad all of the input files to a length of xxx, then the output file will automatically have a length of xxx, so an OUTFIL with OUTREC=(1,xxx) is NOT needed. Thus, you should be able to remove the USING(CTL1) and the //CTL1CNTL DD.

Or am I missing the reason for the OUTREC=(1,xxx) parameter?
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

[videlord]

Frank,

Yes, compare SPLICE only.
The input files are sorted already. No sort needed in PL/I program.

[videlord]

Yes, Frank, the CNTL can be omitted. Thanks.

And one more question:
SPLICE WITHEACH
Can I replace 2 fields of one file?

For example:
BASE ON1
ON1 WITH1
ON1 WITH2A WITH2B
ON1 WITH3

result:
BASE ON1 WITH1 WITH2A WITH3 WITH2B

[Frank Yaeger]

[videlord]

I tested input files with 7M records each, the result shows SPLICE is faster!!!

[Frank Yaeger]