MVSFORUMS.com Forum Index MVSFORUMS.com
A Community of and for MVS Professionals
 
 FAQFAQ   SearchSearch   Quick Manuals   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Date routine performance optimization
Goto page 1, 2  Next
 
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Application Programming
View previous topic :: View next topic  
Author Message
infoman123
Beginner


Joined: 02 Nov 2004
Posts: 57
Topics: 20

PostPosted: Wed Jun 09, 2010 5:55 am    Post subject: Date routine performance optimization Reply with quote

Hi All,

In our system, there are lot of places where lot of cobol logic performing Julian to Greg, Greg to Jul , elapsed days between 2 julian dates, future & back date from a Julian date is performed.
Currently, these routines are performed for each record in the system. We are trying to look for a tuning opportunity here. In such a way that suppose for 1st recod, if these date checks are performed store it somewhere and if any of the later records are with same date use the already calculated value earlier for 1st record and avoid the date routine execution.

For eg
Record 1 date is 2010159 and it has to perform jul to greg, and future date after 1 month, back date before 1 month etc
Record 2 date is 2010160 and it has to perform jul to greg, and future date after 1 month, back date before 1 month etc
Record 3 date is 2010161 and it has to perform jul to greg, and future date after 1 month, back date before 1 month etc
--
--
--
Record 10 date is 2010159 and it has to perform jul to greg, and future date after 1 month, back date before 1 month etc
Record 11 date is 2010160 and it has to perform jul to greg, and future date after 1 month, back date before 1 month etc

What we are looking for is already the calculation for 2010159, 2010160 has performed for record 1 & 2, so these value has to store somewhere and should be used for Record 10 & 11 without recalculating it.

If the date routines are already performed for earlier records, the idea is to re use this for later records without again calculating it.

Can someone help, how can we achieve this with cobol. This will defenitely help us to save our batch window.

Regards
Back to top
View user's profile Send private message
dbzTHEdinosauer
Supermod


Joined: 20 Oct 2006
Posts: 1411
Topics: 26
Location: germany

PostPosted: Wed Jun 09, 2010 6:22 am    Post subject: Reply with quote

well, don't know if you are running IMS, DB2 or vsam.

one thing to consider is that if you attempt, during the run, to 'store' the calculations,
the extra cost of 'storing' the data has to be added to your run time.
then you have the additional overhead of reorg, etc....

there are only 366 potential days in a year. for a 10 year-time frame, this would be 3660 entries someplace (where ever you are going to store).

you have only shown us one calculation. how many types of calculations could be performed on one date? only adding 1 month, only subtracting 1 month.

if it is that simple, I would write a one-off program to populate whatever storage facility (ims, db2, vsam) that you employ. This program would have to take into consideration holidays, etc... But the program would only be run occasionally (yearly to extend for new, and to delete those dates that are too old)

each record/row would contain the julian date, the greg, and the calculated dates.
a simple read would replace the calculation routines.
properly buffered it could possibly be quicker that recalculation.

but again, keep in mind you are replacing potential CPU-boundness with I/O-boundness
_________________
Dick Brenholtz
American living in Varel, Germany
Back to top
View user's profile Send private message
dbzTHEdinosauer
Supermod


Joined: 20 Oct 2006
Posts: 1411
Topics: 26
Location: germany

PostPosted: Wed Jun 09, 2010 8:00 am    Post subject: Reply with quote

still don't know what you are doing, but if the date calculations are so intensive,
sort the file on date and expand the records to include the calculated dates.
that way you could off-load all the date calculation to one program,
since the file is sorted on date,
you could maintain the last calculated and skip the calculations if dates are same.
_________________
Dick Brenholtz
American living in Varel, Germany
Back to top
View user's profile Send private message
Dibakar
Advanced


Joined: 02 Dec 2002
Posts: 700
Topics: 63
Location: USA

PostPosted: Wed Jun 09, 2010 12:59 pm    Post subject: Reply with quote

I beleive logic for Greg to Julian conversion and looking ahead and back a month would be simple. So you can as well copy the logic of the called program in the caller to save time on calls.

Also check your date conversion utility to see if it can be optimised, it may be doing more calls and conversions at backend then you may need.
_________________
Regards,
Diba
Back to top
View user's profile Send private message Send e-mail
infoman123
Beginner


Joined: 02 Nov 2004
Posts: 57
Topics: 20

PostPosted: Thu Jun 10, 2010 1:08 am    Post subject: Reply with quote

Thanks all.

Apologies if my explanation are not clear

Suppose these below are the fields in one record of a file. I am trying to explain it in a simple way but the logic of our program is quite complex to explain here.
Record1
Back to top
View user's profile Send private message
dbzTHEdinosauer
Supermod


Joined: 20 Oct 2006
Posts: 1411
Topics: 26
Location: germany

PostPosted: Thu Jun 10, 2010 3:30 am    Post subject: Reply with quote

you are going to expend resources 'saving calculation results'.

why not sort the file in date order?
_________________
Dick Brenholtz
American living in Varel, Germany
Back to top
View user's profile Send private message
prino
Banned


Joined: 01 Feb 2007
Posts: 45
Topics: 5
Location: Oostende

PostPosted: Thu Jun 10, 2010 5:30 am    Post subject: Reply with quote

The following code converts a date to a Julian Day Number, all arithmetic is integer:

Code:
{***********************************************************************
* CJ:                                                                  *
*                                                                      *
* Convert a date to its Julian Day Number                              *
*                                                                      *
* Original code used:                                                  *
*                                                                      *
* var cj_y: double;                                                    *
* var cj_c: double;                                                    *
*                                                                      *
* cj_y:= dyear + (dmonth - 2.85) / 12;                                 *
* cj_c:= 0.75 * trunc(cj_y / 100);                                     *
* jdn:= trunc(                                                         *
*       trunc(                                                         *
*       trunc(367 * cj_y) - 1.75 * trunc(cj_y) + dday) - cj_c) +       *
*                           1721115;                                   *
*                                                                      *
* Reworked code to use only integer arithmetic:                        *
*                                                                      *
* var c : longint;                                                     *
* var iy: longint;                                                     *
* var ia: longint;                                                     *
* var ib: longint;                                                     *
* var it: longint;                                                     *
*                                                                      *
* begin                                                                *
*   iy:= dyear * 88080 + dmonth * 7340 - 20919) div 240;               *
*   it:= dyear * 1200 + dmonth * 100 - 285;                            *
*   ia:= it div 1200;                                                  *
*   ib:= (iy * 4 - ia * 7 + dday * 4) div 4;                           *
*   c := 3 * (it div 120000);                                          *
*   jdn:= (ib * 4 - c) div 4 + 1721115;                                *
***********************************************************************}

and this does the inverse:
Code:
************************************************************************
* JC:                                                                  *
*                                                                      *
* Convert a Julian Day Number to a date                                *
*                                                                      *
* Original code used:                                                  *
*                                                                      *
* var n: double;                                                       *
* var c: double;                                                       *
*                                                                      *
* n:= 1.0 * jdn - 1721119.2;                                           *
* c:= trunc(n / 36524.25);                                             *
*                                                                      *
* if jdn < 2299161 then                                                *
*   n:= n + 2                                                          *
* else                                                                 *
*   n:= n + c - trunc(c / 4);                                          *
*                                                                      *
* yyyy:= trunc(n / 365.25);                                            *
* n   := n - trunc(365.25 * yyyy) - 0.3;                               *
* mm  := trunc(n / 30.6);                                              *
* dd  := trunc(n - 30.6 * mm + 1);                                     *
*                                                                      *
* if mm > 9 then                                                       *
*   begin                                                              *
*     dec(mm, 9);                                                      *
*     inc(yyyy);                                                       *
*   end                                                                *
* else                                                                 *
*   inc(mm, 3);                                                        *
*                                                                      *
* Reworked code to use only integer arithmetic:                        *
*                                                                      *
* var n: longint;                                                      *
* var c: longint;                                                      *
*                                                                      *
* n:= 10 * jdn - 17211192;                                             *
* c:= (n * 10) div 3652425;                                            *
*                                                                      *
* if jdn < 2299161 then                                                *
*   n:= n + 20                                                         *
* else                                                                 *
*   n:= n + (c - c div 4) * 10;                                        *
*                                                                      *
* yyyy:= (n * 10) div 36525;                                           *
* n   := n - 3650 * longint(yyyy) - 10 * (yyyy div 4) - 3;             *
* mm  := n div 306;                                                    *
* dd  := (n - 306 * mm + 10) div 10;                                   *
*                                                                      *
* if mm > 9 then                                                       *
*   begin                                                              *
*     dec(mm, 9);                                                      *
*     inc(yyyy);                                                       *
*   end                                                                *
* else                                                                 *
*   inc(mm, 3);                                                        *
***********************************************************************}


Both routines are fast but if you were to code them in assembler, replacing the divides by multiplies and shifts, they would be even faster. CJ graciously handles overflowed dates, ie 2010-01-32 is treated as 2010-02-01. Code is Pascal.
Back to top
View user's profile Send private message
Dibakar
Advanced


Joined: 02 Dec 2002
Posts: 700
Topics: 63
Location: USA

PostPosted: Thu Jun 10, 2010 11:35 am    Post subject: Reply with quote

Quote:

Suppose In the program, for each record,
the elapsed days between todays date and process date are calculated to perform a special processing logic. Like if its less than 15 do process 1 and if its > 15 do process 2 etc
if membership date = todays date, the calculate new membership date 1 year future date.
If expire date = todays date, the calculate new expiry date 1 year future date or future 1 month extension date based on some other condition.
If finance date = todays date, calculate the back date of finance due date by 1 month and perform some logic.


Looks like you need only few dates - today, today -15, today + 1 year, today + 1 month, today -1 month. So why not store these 5 dates before processing the records. Then for each record you can compare the date fields with with these dates instead of calling date routines.
_________________
Regards,
Diba
Back to top
View user's profile Send private message Send e-mail
infoman123
Beginner


Joined: 02 Nov 2004
Posts: 57
Topics: 20

PostPosted: Fri Jun 11, 2010 3:58 am    Post subject: Reply with quote

Thanks Prino..

For converting JD to greg and viceversa..dont need this complex logic... basically if we hard code all the equivalent dates in working storage for leap and non leap year.. thts more faster.. our program is coded like that...

Dibakar... as i explained these" - today, today -15, today + 1 year, today + 1 month, today -1 month" but actual program logic is more complex than this. all these date calculation in our program is based on some parameters defined.

basically what i'm looking is something like this
suppose i want find elapsed days between 2 dates date 1 => 2010159 & date 2 => 2010156, then if its caluclated already for rec 1, for for later records if the elapsed days required for date 1 & date 1, the ws-elapsed-days (date1,date2) should give result 3 which is alredy calculated.
Back to top
View user's profile Send private message
Dibakar
Advanced


Joined: 02 Dec 2002
Posts: 700
Topics: 63
Location: USA

PostPosted: Mon Jun 14, 2010 11:31 am    Post subject: Reply with quote

informan123,

I think you need to answer two questions from Dick -

    Why are you not pre sorting on date

    How many date calculations are you expecting to save in memory


Also what is the date window of your calculations. From your questions it looks like the dates are falling in a short window, maybe couple of months from current date, thats why you expect lots of repeatation. If so specify the window for all the date fields that may occur in your calculation.
_________________
Regards,
Diba
Back to top
View user's profile Send private message Send e-mail
jsharon1248
Intermediate


Joined: 08 Aug 2007
Posts: 291
Topics: 2
Location: Chicago

PostPosted: Mon Jun 14, 2010 1:33 pm    Post subject: Reply with quote

What leads you to this conclusion?
Quote:
This will defenitely help us to save our batch window.


I have my doubts. What makes you believe that converting dates is taking so much time? What analysis have you done that makes you believe that eliminating the date conversions is going to save elapsed time?
Back to top
View user's profile Send private message
infoman123
Beginner


Joined: 02 Nov 2004
Posts: 57
Topics: 20

PostPosted: Thu Jun 17, 2010 7:22 am    Post subject: Reply with quote

jsharon1248 wrote:
What leads you to this conclusion?


I have my doubts. What makes you believe that converting dates is taking so much time? What analysis have you done that makes you believe that eliminating the date conversions is going to save elapsed time?


This is made based on looking the actual data in system. Daily there are lot of financial transactions happened are added to system. 80% of the input file from customer reps are on same day. 10% in same date in a week and 5 %in same month and 5% same year. But the logics are generic enough to handle any dates.

so based on this assumption only,if any date calculation is made same somewhere like in a scratch pad and use it as and when needed. If its not calculated perform the normal routine..

Regards
Back to top
View user's profile Send private message
infoman123
Beginner


Joined: 02 Nov 2004
Posts: 57
Topics: 20

PostPosted: Thu Jun 17, 2010 7:25 am    Post subject: Reply with quote

Dibakar wrote:

I think you need to answer two questions from Dick -
    Why are you not pre sorting on date


    How many date calculations are you expecting to save in memory


Also what is the date window of your calculations. From your questions it looks like the dates are falling in a short window, maybe couple of months from current date, thats why you expect lots of repeatation. If so specify the window for all the date fields that may occur in your calculation.


jsharon1248 wrote:
What leads you to this conclusion?


I have my doubts. What makes you believe that converting dates is taking so much time? What analysis have you done that makes you believe that eliminating the date conversions is going to save elapsed time?


We cannot pre sort based on dates, then the whole processing logic needs to be changed.. the file is stored with a key of id number.
Back to top
View user's profile Send private message
jsharon1248
Intermediate


Joined: 08 Aug 2007
Posts: 291
Topics: 2
Location: Chicago

PostPosted: Thu Jun 17, 2010 7:57 am    Post subject: Reply with quote

You really haven't answered the question. Look at it this way. If you eliminated the date conversions all together, how much elapsed time would you save? Now, having said that, if 80% of the transactions are using the same date, why not add a snippet of code to bypass the date conversions if the dates on 2 consecutive records are the same. The converted values will already be in working storage. In your testing phase, add counters to determine how many date conversion are bypassed. Now you're in a position to run a parallel with the prod version to determine how much time you're really saving. Something like this:

Code:
IF INPUT-DATE NOT = WS-DATE
    PERFORM DATE-CONVERSION-PARA
END-IF
Back to top
View user's profile Send private message
Dibakar
Advanced


Joined: 02 Dec 2002
Posts: 700
Topics: 63
Location: USA

PostPosted: Thu Jun 17, 2010 12:43 pm    Post subject: Reply with quote

I am trying to extend the solution propsed by jsharon1248.

infoman123 wrote:
80% of the input file from customer reps are on same day.


infoman123 wrote:
10% in same date in a week


I am guessing your program can determine what are these most occuring date and week are.

Create a table to save date calculations for this date and all the days of the most frequent week. Have the most frequent date as the first row.

Now 90% of calculations are saved without taking two much space. And scanning through them to find your date won't take too much time either.

You may extend this logic to save more date calculations that you think are frequent, put more frequent ones at top.

If you don't have logic to calculate most frequent dates then I beleive solution proposed by jsharon1248 is the only work around.
_________________
Regards,
Diba
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic   printer-friendly view    MVSFORUMS.com Forum Index -> Application Programming All times are GMT - 5 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


MVSFORUMS
Powered by phpBB © 2001, 2005 phpBB Group