View previous topic :: View next topic |
Author |
Message |
videlord Beginner
Joined: 09 Dec 2004 Posts: 147 Topics: 19
|
Posted: Wed Jan 26, 2005 10:34 pm Post subject: Is paralle processing possible in DFSORT utility? |
|
|
If there are several statements of SORT SYSIN, is it possible to make these statements do parallelly?
(Don't want to seperate it to diffirent JCLs and submit)
For example:
//SYSIN DD *
COPY FROM(IN1) TO (OUT1) USING(CTL1NTL)
COPY FROM(IN2) TO (OUT2) USING(CTL2NTL)
COPY FROM(IN3) TO (OUT3) USING(CTL3NTL)
/*
(INx and OUTx are dependent.) |
|
Back to top |
|
 |
kolusu Site Admin

Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
|
Posted: Thu Jan 27, 2005 8:47 am Post subject: |
|
|
Quote: | If there are several statements of SORT SYSIN, is it possible to make these statements do parallelly? (Don't want to seperate it to diffirent JCLs and submit)
|
Videlord,
You cannot process all the operations at the same time without submitting individual jobs.
Kolusu _________________ Kolusu
www.linkedin.com/in/kolusu |
|
Back to top |
|
 |
Frank Yaeger Sort Forum Moderator

Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
|
Posted: Thu Jan 27, 2005 11:23 am Post subject: |
|
|
You're using ICETOOL COPY operators, so it's actually //TOOLIN DD * rather than //SYSIN DD *.
Since you are using three different input data sets, you would have to submit three different jobs in parallel in order to do the processing in parallel.
Note that DFSORT/ICETOOL can do parallel output using a single input file, but that's not the case here. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort |
|
Back to top |
|
 |
dtf Beginner
Joined: 10 Dec 2004 Posts: 110 Topics: 8 Location: Colorado USA
|
Posted: Thu Jan 27, 2005 11:35 am Post subject: |
|
|
Frank,
I think this is a heck of an idea. I wonder if anyone has thought about it before in relation to utilities of this type? If you have functions which are actually independent of each other, they could be set up as subtask........ and potentially gain a fair amount of overlap and improved performance.
Any thoughts on that?
DTF
________
Jaguar XK specifications
Last edited by dtf on Tue Feb 01, 2011 1:41 pm; edited 1 time in total |
|
Back to top |
|
 |
Frank Yaeger Sort Forum Moderator

Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
|
Posted: Thu Jan 27, 2005 1:06 pm Post subject: |
|
|
Actually, DB2 Utilities uses this technique to invoke multiple sorts. It's not a trivial task to manage the resources correctly.
It's not clear if it would be worth it to do this kind of thing for copies since they are normally very fast anyway. _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort |
|
Back to top |
|
 |
superk Advanced

Joined: 19 Dec 2002 Posts: 684 Topics: 5
|
Posted: Thu Jan 27, 2005 3:32 pm Post subject: |
|
|
DTF, I have been using multi-threading logic on batch processing for many years now. It's not unusual. |
|
Back to top |
|
 |
dtf Beginner
Joined: 10 Dec 2004 Posts: 110 Topics: 8 Location: Colorado USA
|
Posted: Fri Jan 28, 2005 10:42 am Post subject: |
|
|
I have also used mutil-tasking in the past. One example of this was a batch system that I worked on which used a parm to set the number of task which would be started to do the work. This was a health insurance claims processing system. As I recall, we determined that 8 task was optimal. Less than that and the job ran longer, and more than that and the job ran longer.
I directed my question at Frank because I wondered if they had thought about incorporating this into the sort product.
________
buy herbalaire vaporizer |
|
Back to top |
|
 |
|
|