MVSFORUMS.com

moyeenkhan · Beginner Joined: 04 Dec 2002 Posts: 64 Topics: 21

What is the best and efficient way to sort 2 million records. Do you think MAXSORT is the way to go?

kolusu · Posted: Wed Feb 16, 2011 12:09 pm Post subject:

moyeenkhan,

Depends on lots of factors.

1. LRECL
2. RECFM
3. Length of the key to be sorted.
4. Format of the key to be sorted.
5. Availability of work datasets/Main storage....
_________________
Kolusu
www.linkedin.com/in/kolusu

papadi · Supermod Joined: 20 Oct 2009 Posts: 594 Topics: 1

Several times a month i sort (well, not personally<g> - this is scheduled) between 8 and 10 million records that are more than 1k each (the whole record is used not a subset - see below). This typically runs in a few minutes. . .

One way to speed up a sort is if the sorted output does not need all of the data in the input records. A subset containing the sort-key(s) and needed data may run exponentially faster than sorting the whole set of data. Not only will this run faster but there will be less need for sortwork and output file dasd (which sometmes encounters space allocation problems).

Lots of times the entire set of data is sorted because it takes so little work to simply "sort it all". The process i mentioned earlier sorts the full data because the data as received is not in the correct sequence for most of the processing that uses this data.
_________________
All the best,

di

Frank Yaeger · Posted: Wed Feb 16, 2011 1:44 pm Post subject:

moyeenkhan,

Which sort product are you using?

Have you tried sorting your 2 million records with a "normal" sort and run into a "problem"?

You really need to give more information before anyone can help you.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

moyeenkhan · Beginner Joined: 04 Dec 2002 Posts: 64 Topics: 21

Many thanks for your time and response.

Here are moe details:
We have Syncsort
The input file is VB - 7508 Bytes
Sort Key details are
OPTION Y2PAST=1951
SORT FIELDS=(45,5,CH,A,94,2,CH,A,132,1,Y2D,A,133,2,PD,A)
RECORD TYPE=V,LENGTH=9674

Currently using MAXSORT

Frank Yaeger · Posted: Wed Feb 16, 2011 4:14 pm Post subject:

I'm a DFSORT guy, not a Syncsort guy, but FWIW, I'd be surprised if MAXSORT was really needed for this. I would think that Syncsort could do this without MAXSORT.
_________________
Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort

moyeenkhan · Beginner Joined: 04 Dec 2002 Posts: 64 Topics: 21

Thanks Frank. I also think I do not need MAXSORT.

Bill Dennis · Posted: Wed Feb 16, 2011 10:59 pm Post subject:

moyeenkhan,

MAXSORT is not for efficiency, it is for constrained SORTWORK space. The intermediate sort and merge phases require more resources. It is best reserved for sorting enormous files when the required work space cannot be allocated.
_________________
Regards,
Bill Dennis

Disclaimer: My comments on this foorum are my own and do not represent the opinions or suggestions of any other person or business entity.

moyeenkhan · Beginner Joined: 04 Dec 2002 Posts: 64 Topics: 21

Thanks Bill.