MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Effect on CPU time - DB2 prog with cursor
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Application Programming
View previous topic :: View next topic  
Author Message
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Wed Jul 05, 2006 4:44 am    Post subject: Effect on CPU time - DB2 prog with cursor Reply with quote

Hi,

I have a program that selects daily data from a DB2 table using Cursor and formats some fields of data and writes them to an O/P file. In other words it extracts the data from DB to file. Daily data can go upto 32K.

CPU time for it's execution is 2 min 27 sec. What can be the reason for the CPU time to go in min? Is it because of the cursor?

Is there any other way so that i can reduce the CPU time for this Job.

Thanks,
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Wed Jul 05, 2006 7:04 am    Post subject: Reply with quote

deepatred,

Check this explanation about CPU time

http://mvsforums.com/helpboards/viewtopic.php?p=948#948

In your case you need to check if your Cursor is using an index or not. Also another alternative is to use DSNTIAUL to unload and reformat the output to your needs.

Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
bauer
Intermediate


Joined: 10 Oct 2003
Posts: 315
Topics: 49
Location: Germany

PostPosted: Thu Jul 06, 2006 1:17 am    Post subject: Reply with quote

deepatred,

- check your cursor using db2 explain command
- use any monitoring tool (omeganmon ....)
- check your program logic (loops .....???)


regards
bauer
Back to top
View user's profile Send private message
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Thu Jul 06, 2006 2:55 am    Post subject: Reply with quote

Hi Kolusu,

Thanks for response.


Quote:
In your case you need to check if your Cursor is using an index or not.


Cursor is using indexes in Where and order by clause, only two extra fields are present in where clause that not part of indexes.

Quote:
Also another alternative is to use DSNTIAUL to unload and reformat the output to your needs.


I tried both logics in test region to see the diff. For 180 records in DB (test region) these are some the statistics.
1) Actual cursor program.
It took 00.63 sec of CPU.

2) Another job with one step for DB extract using DSNTIAUL and second step for executing program with that extract file as input and formatting and creating o/p file.
step1 to extract took 00.12 sec
step2 for executing prog to format data: 00.61 sec.

But even this breaking of logic is somehow again increasing the time of job.
Sad

I was going through the link that you provided. According to that info during file processing program issues EXCP calls and is eligible to be "swapped out" or "put on hold".
So does it mean that increase in no. of files used in a program increases it's chance of being swapped out? hence increase in processing time.

Since this happens during each record processing, increase in no of records of file may also increase it's chance of being swapped out.

How DB2 program are processed?

Thanks.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Thu Jul 06, 2006 5:06 am    Post subject: Reply with quote

Quote:

2) Another job with one step for DB extract using DSNTIAUL and second step for executing program with that extract file as input and formatting and creating o/p file.
step1 to extract took 00.12 sec
step2 for executing prog to format data: 00.61 sec.


Deepatred,

In which language is the Pgm to reformat the data coded? Can you explain the rules of reformating? If so may be I can suggest something.

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
bauer
Intermediate


Joined: 10 Oct 2003
Posts: 315
Topics: 49
Location: Germany

PostPosted: Thu Jul 06, 2006 5:33 am    Post subject: Reply with quote

deepatred,

if you are talking about "Cursor is using indexes " the next questions are:

- Matchcolumns?
- Clusterindex ?
- Index Scan? Non Matching Index Scan ?
- Results of EXPLAIN?


regards Bauer
Back to top
View user's profile Send private message
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Thu Jul 06, 2006 11:05 pm    Post subject: Reply with quote

Hi Kolusu,

Program is in Cobol. The processing that program does is somewhat like this.


for each record.
if field-1 not = 'HH'
increment counter count-1
if field-1 = 'CC'
Revert signs of 6 fields.
Feild-2 = field-2 * -1
Feild-3 = field-3 * -1
.
.
Feild-7 = field-7 * -1

split field-8 into 2 parts using a group variable. output file has it as
field-8A and field-8B
Write rest of the fields as it is to output file1.

file-2
1) write total record processed count.
2) write total count of field-1 not = 'HH' case: count1

this second file is a sort of report to be used latter for some processing by some other Job. It contains total records that were fetched and count1.

Thanks.
Back to top
View user's profile Send private message
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Thu Jul 06, 2006 11:06 pm    Post subject: Reply with quote

Hi Bauer,

Thanks for the reply.
Actually i am somewhat new to mainframe and DB2, so not familiar to all those terms.
Could you please let me know some more details. Where to find those info and how these things affect
the performance of query.

Thanks.
Back to top
View user's profile Send private message
bauer
Intermediate


Joined: 10 Oct 2003
Posts: 315
Topics: 49
Location: Germany

PostPosted: Fri Jul 07, 2006 1:11 am    Post subject: Reply with quote

deepatred,

the topic "how to use DB2 explain command" was dicussed several times in this forum. Pls use the search function to find the link.

But in general the influence on performance regarding the index structure, type on index, cluster / non cluster index, valid static data ..... require some skill and experience. So if you are not familiar with some general concepts the results of the DB2 explain Command wounn't help you. Because of the explain command (and in addition catlog information) will only tell you what for example "an cluster index with matchcolumns 2 is used".

So my proposel for the future: Look for an training / basic information about the general concepts of DB2 (and other database systems.......). Well it's hard, but witthout knowing your select statement, the databasestructure and the statistic data nowbody will be able to find out wether your select is the problem or the bad performance has any other reasons.

The background of my post was only, to give some ideas, not to explain general database concepts and the implementaion in ibms db2.

regards,
bauer
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Fri Jul 07, 2006 7:27 am    Post subject: Reply with quote

deepatred,

I don't think you really need a cobol program for formatting the output. you can use an utility to format the input or even the unload utility can do that itself. Post your cobol layout of the table and also specify the fields that need to have the sign reverted and split fields. A sample input and desired output also would help.

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Tue Jul 11, 2006 4:40 am    Post subject: Reply with quote

Hi Kolusu,

Code:
01  RS01.                           
   10 RS01-FIELD1  PICTURE  X(24).       
   10 RS01-FIELD2  PICTURE  S9(3)     COMPUTATIONAL-3.                   
   10 RS01-FIELD3  PICTURE  S9(8)     BINARY.                           
   10 RS01-FIELD4  PICTURE  XX.           
   10 RS01-FIELD5  PICTURE  S9(9)     COMPUTATIONAL-3.                   
   10 RS01-FIELD6  PICTURE  S9(9)V99 COMPUTATIONAL-3.                   
   10 RS01-FIELD7  PICTURE  S9(7)V99 COMPUTATIONAL-3.                   
   10 RS01-FIELD8  PICTURE  S9(7)V99 COMPUTATIONAL-3.                   
   10 RS01-FIELD9  PICTURE  S9(7)V99 COMPUTATIONAL-3.                   
   10 RS01-FIELD10 PICTURE  S9(9)V99 COMPUTATIONAL-3.                   
   10 RS01-FIELD11 PICTURE  S9(7)V99 COMPUTATIONAL-3.                   
   10 RS01-FIELD12 PICTURE  X.           
   10 RS01-FIELD13 PICTURE  X(5).         
   10 RS01-FIELD14 PICTURE  X.           
     


Field1 to be split into X(13) and S9(11)- length 6 (comp-3).

Field5, 6, 7......11 sign to be reversed.
We don't have DFSORT utility here.

Thanks.
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Tue Jul 11, 2006 7:59 am    Post subject: Reply with quote

Deepa,

If you don't have DFSORT then you may not be able to use IFTHEN and OVERLAY which would make job easier and all calculations can be done within a single step. With syncsort version you have you need two steps. try this job

Code:

//STEP0100 EXEC PGM=SORT                         
//SYSOUT   DD SYSOUT=*                           
//SORTIN   DD DSN=YOUR INPUT FILE
//            DISP=SHR                           
//CC       DD DSN=&T1,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//HH       DD DSN=&T2,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//REST     DD DSN=&T3,DISP=(,PASS),SPACE=(CYL,(X,Y),RLSE)
//RPT      DD DSN=YOUR REPORT COUNT FILE,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(TRK,(1,1),RLSE)
//SYSIN    DD *                                   
  SORT FIELDS=COPY                             
  OUTREC FIELDS=(01,76,                       
                 77:C'1',                     
                 78:31,2,CHANGE=(1,C'HH',C'0'),
                 NOMATCH=(C'1'),               
                 SEQNUM,8,ZD)                 
                                               
  OUTFIL FNAMES=CC,                           
  INCLUDE=(31,2,CH,EQ,C'CC'),                 
  OUTREC=(01,13,                               
          14,11,ZD,PD,LENGTH=6,               
          25,08,                               
          -1,MUL,33,5,PD,PD,LENGTH=5,         
          -1,MUL,38,6,PD,PD,LENGTH=6,         
          -1,MUL,44,5,PD,PD,LENGTH=5,         
          -1,MUL,49,5,PD,PD,LENGTH=5,         
          -1,MUL,54,5,PD,PD,LENGTH=5,         
          59,18,                               
          5C'0',                               
          79,8)                               
                                               
  OUTFIL FNAMES=HH,                           
  INCLUDE=(31,2,CH,EQ,C'HH'),                 
  OUTREC=(01,13,                               
          14,11,ZD,PD,LENGTH=6,               
          25,52,                               
          SEQNUM,5,ZD,                         
          79,8)                               
                                               
  OUTFIL FNAMES=REST,SAVE,                     
  OUTREC=(01,13,                               
          14,11,ZD,PD,LENGTH=6,               
          25,52,                               
          5C'0',                               
          79,8)                               
                                               
  OUTFIL FNAMES=RPT,                                     
  REMOVECC,NODETAIL,                                     
  TRAILER1=('TOTAL RECORDS PROCESSED              : ',   
             TOT=(77,1,ZD,M11,LENGTH=8),/,               
            'TOTAL RECORDS OF RS01-CNOPR NOT = HH : ',   
             TOT=(78,1,ZD,M11,LENGTH=8),                 
             80:X)                                       
/*                                                       
//STEP0200 EXEC PGM=SORT                                 
//SYSOUT   DD SYSOUT=*                                   
//SORTIN   DD DSN=&T1,DISP=SHR                           
//         DD DSN=&T2,DISP=SHR                           
//         DD DSN=&T3,DISP=SHR                           
//SORTOUT  DD DSN=YOUR OUTPUT FILE,
//            DISP=(NEW,CATLG,DELETE),
//            UNIT=SYSDA,
//            SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD *                                           
  SORT FIELDS=(77,8,CH,A)                                 
  OUTREC FIELDS=(1,76)                                   
/*



Hope this helps...

Cheers

Kolusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Fri Jul 14, 2006 1:07 am    Post subject: Reply with quote

Hi Kolusu,

Thanks a lot for this solution it's working out and taking only .10 sec Smile also got to learn some new features of syncsort that i had not used till now.

But there is one problem that i could not figure out, file REST is not getting any record? Those non 'HH' and 'CC' records are not getting written in REST file and somehow getting discarded and thus the final output file is not getting those records.

Also why 5 digit SEQNUM is being generated in file 'HH'? As far as i can
see we are not using that anywhere.


Thanks,

Regards,
Deepa
Back to top
View user's profile Send private message
kolusu
Site Admin
Site Admin


Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose

PostPosted: Fri Jul 14, 2006 4:15 am    Post subject: Reply with quote

Quote:

But there is one problem that i could not figure out, file REST is not getting any record? Those non 'HH' and 'CC' records are not getting written in REST file and somehow getting discarded and thus the final output file is not getting those records


Are you sure you have the keyword SAVE on the OUTFIL of rest? If so all the non 'HH' and 'CC' WILL be copied to hat DD name.

Quote:

Also why 5 digit SEQNUM is being generated in file 'HH'? As far as i can
see we are not using that anywhere.


Isn't that one of your requirement , if the code is 'cc' then increment the counter?

KOlusu
_________________
Kolusu
www.linkedin.com/in/kolusu
Back to top
View user's profile Send private message Send e-mail Visit poster's website
deepatred
Beginner


Joined: 29 Nov 2005
Posts: 19
Topics: 6

PostPosted: Fri Jul 14, 2006 5:43 am    Post subject: Reply with quote

YEs i have written exactly similar to what you have given.

Code:

//STEP0010 EXEC PGM=SORT                                       
//*                                                           
//SORTIN   DD DSN=MYDATA.TEMP.RS01.IFILE1,DISP=SHR             
//*                                                           
//CC       DD DSN=MYDATA.TEMP.RS01.OFILECC,                   
//            DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(1,1),RLSE)   
//HH       DD DSN=MYDATA.TEMP.RS01.OFILEHH,                   
//            DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(1,1),RLSE)   
//REST     DD DSN=MYDATA.TEMP.RS01.OFILER,                   
//            DISP=(NEW,CATLG,DELETE),SPACE=(CYL,(1,1),RLSE)   
//RPT      DD DSN=MYDATA.TEMP.RS01.RPT1,                     
//            DISP=(NEW,CATLG,DELETE),                         
//            SPACE=(80,(300,100),RLSE),                       
//            LRECL=80                                         
//*                                                           
//SYSIN  DD *                                                 
  SORT FIELDS=COPY                                             
  OUTREC FIELDS=(01,76,                                       
                 77:C'1',                                     
                 78:31,2,CHANGE=(1,C'HH',C'0'),               
                 NOMATCH=(C'1'),                 
                 SEQNUM,8,ZD)                     
                                                 
  OUTFIL FNAMES=CC,                               
  INCLUDE=(31,2,CH,EQ,C'CC'),                     
  OUTREC=(01,13,                                 
          14,11,ZD,PD,LENGTH=6,                   
          25,08,                                 
          -1,MUL,33,5,PD,PD,LENGTH=5,             
          -1,MUL,38,6,PD,PD,LENGTH=6,             
          -1,MUL,44,5,PD,PD,LENGTH=5,             
          -1,MUL,49,5,PD,PD,LENGTH=5,             
          -1,MUL,54,5,PD,PD,LENGTH=5,             
          59,18,                                 
          5C'0',                                 
          79,8)                                   
                                                 
 OUTFIL FNAMES=HH,                               
 INCLUDE=(31,2,CH,EQ,C'HH'),                     
 OUTREC=(01,13,                                               
         14,11,ZD,PD,LENGTH=6,                                 
         25,52,                                               
         SEQNUM,5,ZD,                                         
         79,8)                                                 
                                                               
 OUTFIL FNAMES=REST,SAVE                                       
 OUTREC=(01,13,                                               
         14,11,ZD,PD,LENGTH=6,                                 
         25,52,                                               
         5C'0',                                               
         79,8)                                                 
                                                               
 OUTFIL FNAMES=RPT,                                           
 REMOVECC,NODETAIL,                                           
 TRAILER1=('TOTAL RECORDS PROCESSED              : ',         
            TOT=(77,1,ZD,M11,LENGTH=8),/,                     
           'TOTAL RECORDS OF RS01-CNOPR NOT = HH : ',         
            TOT=(78,1,ZD,M11,LENGTH=8),                       
            80:X)                                 
/*                                               
//SYSERR   DD SYSOUT=*                           
//SYSOUT   DD SYSOUT=*                           
//SYSPRINT DD SYSOUT=*                           
//SYSUDUMP DD SYSOUT=*                           
//*                         



Oh that was a working storage counter and used later to write count in the second file. But that is fine, i just thought if putting that seq no. had any other significance.

Thanks,

Regards,
Deepa
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Application Programming All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group